RPi and the VistaCNC P4S pendant

Hi folks,
I'm trying to integrate a VistaCNC P4S pendant into my machine running on an RPi 4 with LinuxCNC 2.8.2. I followed the instructions included with the firmware download, and got it to work in the sense that the display shows the selected axis with the DRO reading. I'm seeing that the entire RPi starts to act sluggish, with laggy/jerky mouse movements and a keyboard that randomly ignores key-up and key-down events.  For example, I was using vim to edit a hal file and noticed that sometimes it would repeat the last pressed key over and over, until I pressed the same key again. When jogging the machine using the P4S in velocity mode, the axis sometimes continues to move after I stop turning the jog wheel (I have to turn it backwards to stop it). I noticed that the DRO updates on both the pendant as well as on screen were jerky (would not update for a quarter second, then suddenly 'catch up'). If I run something in the background like glxgears, it continues to update even when the DRO freezes.

All the odd behavior stops completely when I shut down LinuxCNC, or if I comment out the lines in the hal file that load the p4s driver. From what I can tell, the vc-p4s.hal loads the vc-p4s binary as a user component. Am I doing something to cause it to poll the USB bus too often?

I am on kernel 4.19, with PREEMPT-RT.

Thanks for the response. The VistaCNC pendant is USB based. /var/log/kern.log shows a few disconnect messages, but it wasn't a continuous stream.

Side note: since the onboard wifi wasn't performing well, I plugged in a USB wifi dongle (on the powered USB hub) and that caused the jerky mouse and dropped key events to be more frequent. I also tried out a newer kernel that was posted here on the forum (as pre-made ISO) and that seemed to behave much worse in terms of dropped keys.

Some searching online indicated that RPis can have issues with low supply voltage, and being unable to power USB devices with the usual 500 mA. I used a powered USB hub, but that didn't solve the issue. I also adjusted the supply voltage to the Pi up to 5.2V (I'm using a Mesa 7c81, and a 2A 24V to 5V DC/DC). I will swap out the DC/DC for a different power supply, I have one on order.

I've sort of given up on the VistaCNC pendant, and I've ordered a 7i73 which seems like a much better option. In testing this morning with the USB pendant, I managed to crush my X axis limit switch!

I replaced the DC/DC with a Mean-well AC to DC 5V/2A DIN-rail power supply. I biased the supply up a bit to 5.2V, and verified that the voltage on the RPi test points was > 5V. The pendant still caused the same issues as before, with missing keys, jerky mouse movement. No messages were logged in kern.log.