I've tried single ended... but it didn't work.
I've applied 7V tot the encoder, (from the field power) Now it looks like it is working.

In "Show Hal Configuration" the hm2_5i25.0.encoder.00.count pin shows +-2000 counts/turn. This is probably the encoders 500 pulses multiplied by 4. (I bought I read that somewhere.. the count is pulsesx4)