Am I wasting my time? Raspberry Pi 4B - 7C81

There are a lot of screenshots and data I can post, but I'll start out keeping this brief. I followed @alangibson's post on how to Install LinuxCNC 2.9 on Raspberry Pi 4B with the Preempt-RT kernel. Excellent post that really helped. Since then I've been reading online about Linux jitter online and trying to get it as low as possible.

My setup may be a little different in that I'm using a laptop as the keyboard and display for the Raspbery Pi 4B. I have an ethernet crossover cable between them, set up static IP addresses, and ssh -X into the Pi. I've disabled everything I can on the Pi including its display manager since it doesn't need it.

If I let the latency test or histogram run for a couple hours with two glxgears running The best it does is  a bit under 30us. Adding things I've read online to "improve" jitter increases it to  just over 60us. (BTW, it seems like the number from latency-test matches the + and - numbers from latency-histogram. Would have thought the latency-test number would be the plus to minus spread) The stuff I've read online about isolcpus (deprecated) and cset (over-ridden by systemd) is fairly confusing. Bottom line is I've been fiddling with this for a few weeks in my spare time and I'm getting weary of making no progress. Since the 7C81 board's FPGA supports stepgen, does any of this matter anyway? If the Servo thread has 65us jitter will it even matter? I want to move forward with the CNC build. 

Again, I'm happy to post screenshots and settings if it helps.

Thanks for any feedback.

Gene

Do you get latency errors from lcnc?

Honestly, just download a linuxcnc image, boot it up, make chips
disable sound, bluetooth, wifi, whatever you dont need.

will it matter? maybe? I havent had any issues before, and its been a few years now.

The jitter is basically unimportant with a 7C81 unless its so bad you
get real time errors at the normal servo thread rate.

Isolcpus is not deprecated. Its purpose is to isolate some cores to only be used by real time threads.
Our testing showed improvements with isolcpus=2,3

Thanks for confirming that I'm chasing my tail and wasting time.

There is a lot of conflicting information on the internet regarding how to reduce jitter in Linux. That typically means black magic time sink that varies greatly from setup to setup.

Moving forward.

@rodw -This is from kernal.org
 

Best,

Gene

Interesting. Yet in other places kernel.org has docs for isolcpus without mentioning that. Linuxcnc is not a complex applictation, and we are users not developers. isolcpus generally gives performance improvements as it isolates some cores that are basically exclusively available  for Linuxcnc's real time threads.

In recent testing this week we saw latency drop from about 90k to 50k by adding isolcpus=2,3 for the Pi.
If you are getting latency of 60k, move on and worry if it doesn't work. I'm sure it will be fine.

It's a bit involved, but we put a guide together here for inclusion ito the Linuxcnc docs. There are a few more tweaks there you can try...
github.com/rodw-au/linuxcnc/blob/rodw-ge...sing-debian-bookworm
But don't bother with this unless yours stops working...

​​​​
 

With the dietpi image I setup I was averaging around 40us with 5 x glxgears running and no real overclocking
So it sounds about right although I think it'll vary based on the kernel version.
6.3 onwards for the newer ones, and maybe 5.4 / 5.10 for the older ones, with the ones inbetween possibly being worse.
As others have said though I don't think it should matter if you're using a mesa card.

My understanding is there's normally two threads, the base thread and the servo thread.
Without a mesa card (such as using the GPIO pins to drive stepper motors directly) the RPI has to do the base thread and generate lots of steps very quickly
With a mesa card that part of the workload is offloaded so the Pi has less to worry about and just manages the servo thread.

If you really want to push things, I think with the newer Pi4's you can run them at 1800 instead of 1500 for the frequency
but it's probably not worth it if there's a mesa card in use
www.raspberrypi.com/news/bullseye-bonus-1-8ghz-raspberry-pi-4/

I did see the same thing in so far as isolcpus being depreciated when I was digging around for options to try out.
I think it's because once it's set at boot time it can't then be changed later on.

cpuset allows you to dynamically turn stuff on and off whenever you want at runtime so is more flexible.
www.spinics.net/lists/linux-rt-users/msg25176.html
www.codeblueprint.co.uk/2019/10/08/isolc...eprecated-kinda.html
stackoverflow.com/questions/11111852/how...eduling-threads-onto

However I couldn't get it to work as far as linuxcnc.
I think cpuset shield is supposed to do the same job as isolcpus at runtime
But when I tried it the historgram wouldn't run, so maybe a linuxcnc bug as far as supporting shielded cpu sets, or maybe it's the wrong method.
 

I think isolcpus is the right tool for Linuxcnc.
I have been doing a bit of kernel tracing with mentoring from a couple of guys from the RT kernel team.
Linuxcnc (setup for Mesa) only uses one thread so it picks one of the isolated CPUs.
We never need to turn it on and off. I think cpuset would need implementation in the code.

Probably using cpuset when starting the threads  in the source only while Linuxcnc is in operation could improve performance if it's not in use
But moving 69 threads seems wrong!

Here are the latency results. Both are with two glxgears processes running.
 

 

Good enough.

Perfect! Move on to the next challenge.