Raspberry Pi 4 with PlasmaC

sir did you try to isocpu 0,2,3 as then internal Rasp are still running on 0 but RT will use 1 as standalone this gives you (might) a mutch better latency

Still getting some hm2/hm2_7i96.0: error finishing read

Been working with home switches on X / Y and during home all I have esc Z as it is not connected yet so maybe that can create the issue.

Have changed SERVO_PERIOD = 2000000

You have _much_ worse latency than I do on my RPI4 .
This might be a setup issue

my cmdline.txt contains:

isolcpus=1,2,3 idle=poll acpi_irq_nobalance noirqbalance

and config,txt contains

force_turbo=1

Will do some tests tomorrow

Is the Latency-Histogram disk intensive as I am running on a µSD card?

Changes made here is
turbo_mode = 1
isolcpus=0,2,3

The tendency is the same even with isolcpus=1,2,3

So is this a matter of slow disk = µSD card?