Latency, error finishing read, and IRQ affinity

I thought I had, I edited the section with the IP address to my card IP address of 10.10.10.11. Obviously I’ve made a mistake, I’ll go back through it again to see.

Yeah, A bit of error/status reporting would improve that script

I think I see my mistake, I only changed the second instance (line 145) to 10.10.10.11 and overlooked line 117. Sorry for the trouble and thanks for the help!

I knew it was something I had missed. Odd that the old debian 10 vanilla install just worked, but the debian 12 was so finicky about having the cpu isolated and irq pinned.

Yes, I think  some kernel change made Ethernet latency more sensitive to IRQ pinning

If I rerun rt_setup.sh, it is not updating the two files in /etc/linuxcnc and manually editing the lcnc_setirqaffinities.sh to have the correct IP does not result in the IRQ being pinned to the correct CPU (2 instead of 8, still). So looks like I still have some investigations to do, but at least I now know what needs to be accomplished.

Thanks for the script! Had been fighting this issue for a few days with errors talking to a mesa board on a N100 that I was setting up. Worked great on once I fixed the same issue Swag had.  My static IP had been 10.10.10.11.  Changed it to 10.10.10.1 and the ping to 10.10.10.10 was good, previously I was getting random 3ms spikes.

ping 10.10.10.10 -i .02
...
325 packets transmitted, 325 received, 0% packet loss, time 8151ms
rtt min/avg/max/mdev = 0.080/0.101/0.127/0.007 ms

 

Hi! 
I am working with a custom ethernet interface and think I really need this IRQ management. Unfortunately I have not been able to get what's described here to work after quite a few hours beating my head on the desk and rereading ALL these posts several times. I've tried a number of things but I don't know much about scripting. I'm probably missing something obvious.

If someone has debugging suggestions or can tell me what other information to post here I would really appreciate it.

The obvious things:
I did install irqbalance via Synaptic. I did change IP to 10.0.0.1 two places in rt_setup.sh--that's the address in interfaces file.

What happens:
If I run rt_setup.sh it says it is creating the affinity file, and it ends up in /etc/linuxcnc as expected with permissions correct. But after restart I seem to end up with ethernet interrupts always on cpu5 where I would expect cpu7.

The script comes up with isolcpus 6,7 in grub, which is what I determined months ago and added manually and seems like it should be correct.

my /etc/network/interfaces (after removing commented out lines and loopback):
Significant things I tried:
Running test script "pinirq_2024-05-12.txt" with parameter enp8s0f0 displays expected results. But watching interrupts stills shows the nic IRQs on cpu5.

I tried using NIC=`enp8s0f0` in place of both awks in rt_setup.sh but that does not work either (note the quotes I used!).

I did have loopback lines and old (commented out) iface and address lines in interfaces file. I removed them and reran everything as I'm not sure how picky that grep is.


section from /proc/interrupts:

Is it possible /etc/default/irqbalance is not being used and thus not reaching our lcnc_irqbalancepolicy.sh file? I don't understand how that process is supposed to work...

I downloaded lcnc_irqpolicy.sh.txt.
My NIC is "enp4s0" and Mesa ip=10.10.10.10.
I made the modification in the above file and modified file attached. Please check whether any further correction required or it is ok.
============================================================================================================
NIC=`awk 'BEGIN{nic="enp4s0 "} {if ($1=="iface") {tmpnic = $2} if ($1=="address") {if ($2 == 10.10.10.10 ) {nic = tmpnic}}} END{print(nic)}' /etc/network/interfaces`
============================================================================================================

In the interest of contributing something and not solely sucking time asking questions about problems that may well not actually be LinuxCNC's problem...

I've got a Celeron J6412 fanless PC with integrated graphics running my mill. Arch Linux, kernel 6.10.2-rt14-arch1-2-rt from the official Arch repo. Jitter is ~32000 ns- plenty good for Mesa usage, longer than I'd like. That's the result of following this thread's guidance and the usual disabling of power management in the BIOS.

My actual contribution is around video drivers. Re jitter, fbdev works great (albeit slow), the legacy Intel and KMS/modesetting drivers were terrible- latency well over 100000 ns. I'm guessing that's why you see the "nomodeset" kernel option listed. In my case, I found that disabling glamor in xorg.conf lets you use modesetting with seemingly (placebo effect?) better performance, but without any hit to jitter: