Going from Devian 10 to 12 has created problems

For some time now I've enjoyed a stable system created using the LinuxCNC 2.8.2 Debian 10 PREEMPT-RT ISO.   But now that I've upgraded to Debian 12 (I wanted to install a package unrelated to LinuxCNC, and the Debian 10 repository is no longer supported), things have fallen apart.  After the transition, LinuxCNC was still present, but it would not launch.  I assumed that this was because it had run in kernel space in my original system and so was treated specially (e.g. executable not in "bin").  Was this so?  In any case I installed linux-uspace.  It launches as expected, but unfortunately produces a string of error pop-ups:  "Smart Serial Port 0: Dolt not cleared from previous servo thread", "Error finishing read", and so on.

The hardware and BIOS are exactly as they were when things worked perfectly. LinuxCNC is 2.9.5 and the kernel is 6.1.0_39_rt_amd64.  Pinging the Mesa card gives consistent times of .125msec.  The latency looks generally  reasonable, but the histograms show a significant number of extreme outliers (???).

 

Can anybody help?  I feel that its probably a case of some option not being set correctly (i.e. the kernel isn't really pre-empting, LinuxCNC doesn't have the highest priority, or something such).  What might be missing somewhere in some configuration file?

There's also a little matter of the Axis window not maximizing, but this is a relatively minor concern.

 

1. What Ethernet chip does the host PC have?

2. What is the result of running:

sudo chrt 99 ping -i .001 -q -c 60000 [fpga_card_ip_address]

[fpga_card_ip_address] = 10.10.10.10 or 192.168.1.121 depending on card setup

The command will run for 1 minute and print Ethernet timing statistics

I ask about the Ethernet chip type because more recent kernels have a Realtek driver
that has terrible real time performance and must typically be replaced by the DKMS
driver.

There are quite a lot of optimisations required with the later kernels (Debian 12-13). I have documented them in a PDF file available here
erpx.au/shop/trixie-linuxcnc-debian-trix...tion-pdf-download-48
There is a nominal AUD $2.00 fee for this info... Sorry but it was the only way I had to store a file.
While it refers to Debian 13 Trixie, it also is applicable to Debian 12 Bookworm.

I will say that feedback from users I've received says 2 core PCs are marginal for Debian 12-13 but some have got them working with the optimisations I describe. I have never used < 4 core PC's

1/. The computer has a NEX 604 motherboard and uses a dual Realtek 8111E PCI Express Gigabit Ethernet chip.

2/.  

I'm guessing that the 8ms maximum tells you all that you need to know?

 

Yes, that sounds like the standard Realtek driver issue (R8169)
You need to install the r8168-dkms driver

I've installed the r8168 driver.

The bad news is that the ping test you suggested now looks even worse. I tried it three times and in each case the maximum value was around 26ms. And the latency histogram now shows even more pronounced disconnected "peaks" above 20usec and below -20usec.

The good news is that despite the above LInuxCNC now seems to work. I tried launching it three times. On one occasion I got an initial "unexpected delay - this message will not be repeated" pop-up, but apart from this there were no error messages at all, and I successfully had the machine home all axes several times. Either I got very lucky or things have improved considerably.

I'm left very confused (and not very confident).....

Are you sure you are using the DKMS driver?

Can you run this script to verify the driver name:

 

download

chmod +x edrivers.txt
 
./edrivers.txt
 

running your script gives:

enp1s0 r8168
enp1 r8168

I've also tried lspci -nnk | grep -iA3 net. For both ethernet ports this gives

kernel driver in use: r8168
kernel modules: r8168

I did specify DKMS in installing the package, but after this everything became plain "r8168" (????)

You should apt remove r8168 driver or it will take that one.
Best to check though, I might not remember this right.

Hakan wrote:

You should apt remove r8168 driver or it will take that one.
Best to check though, I might not remember this right.

No keep it. It is the correct driver for the R8111
Ref: packages.debian.org/trixie/r8168-dkms

Never remove the driver, always purge it as mentioned on that link

As I said 2 cores may not be enough anymore