Latency, error finishing read, and IRQ affinity
- Mecanix
- Offline
- Platinum Member
Less
More
- Posts: 446
- Thank you received: 216
10 Feb 2024 18:43 #293024
by Mecanix
Ah. You actually did mentioned, missed it, sry.
What's the required voodoo to verify/debug besides $ watch -n1 -d cat /proc/interrupts ?
Replied by Mecanix on topic Latency, error finishing read, and IRQ affinity
It should be run from /etc/rc.local and not as a policyscript for irqbalance.
Ah. You actually did mentioned, missed it, sry.
$ sudo nano /etc/default/grub
#GRUB_CMDLINE_LINUX_DEFAULT="quiet"
GRUB_CMDLINE_LINUX_DEFAULT="quiet isolcpus=1"
$ sudo update-grub
$ sudo nano /etc/rc.local
-----------------------------------------------------
#!/bin/sh -e
#
# IRQ Affinity by mozmck
sh '/etc/irqbalance.d/lcnc_irqpolicy.sh'
exit 0
-----------------------------------------------------
$ sudo chmod +x /etc/rc.local
$ sudo systemctl daemon-reload
$ sudo systemctl status rc-local
What's the required voodoo to verify/debug besides $ watch -n1 -d cat /proc/interrupts ?
Please Log in or Create an account to join the conversation.
- rodw
- Away
- Platinum Member
Less
More
- Posts: 10711
- Thank you received: 3534
10 Feb 2024 19:01 #293025
by rodw
Replied by rodw on topic Latency, error finishing read, and IRQ affinity
If this is stickied now, is this the right section? It might be better in Installing linuxcncMade this sticky, thank you.
The following user(s) said Thank You: seuchato
Please Log in or Create an account to join the conversation.
- Mecanix
- Offline
- Platinum Member
Less
More
- Posts: 446
- Thank you received: 216
10 Feb 2024 19:16 #293026
by Mecanix
One thing for sure, this hack works flawlessly for =>4 cores cpus. Remarkable enhancement I'd like to add. Recommend!!!
Replied by Mecanix on topic Latency, error finishing read, and IRQ affinity
(my take:) I wouldn't bother spending time on those expired dual-core dinosaurs, @mozmck. As soon as you "isolcpus=1" this in grub you instantly lose irqbalance. The service won't even start regardless if the script is ran from a policy or by using rc.local custom/adapted script from rc.local.I haven't tested it myself yet, but I know I have a dual-core or two around here that I can try it on. Probably be next week or so before I can find time to do that.
One thing for sure, this hack works flawlessly for =>4 cores cpus. Remarkable enhancement I'd like to add. Recommend!!!
The following user(s) said Thank You: seuchato, tommylight
Please Log in or Create an account to join the conversation.
- tommylight
- Away
- Moderator
Less
More
- Posts: 19382
- Thank you received: 6490
10 Feb 2024 19:53 #293033
by tommylight
Replied by tommylight on topic Latency, error finishing read, and IRQ affinity
That or "driver boards", what would be more appropriate?If this is stickied now, is this the right section? It might be better in Installing linuxcnc
Please Log in or Create an account to join the conversation.
- cmorley
- Offline
- Moderator
Less
More
- Posts: 7762
- Thank you received: 2046
10 Feb 2024 20:37 #293035
by cmorley
Replied by cmorley on topic Latency, error finishing read, and IRQ affinity
Should probably be added to official docs..
The following user(s) said Thank You: seuchato, tommylight
Please Log in or Create an account to join the conversation.
- mozmck
- Offline
- Administrator
Less
More
- Posts: 64
- Thank you received: 55
11 Feb 2024 01:20 - 29 Feb 2024 21:11 #293049
by mozmck
Replied by mozmck on topic Latency, error finishing read, and IRQ affinity
Here is a script which will do the following:
There are a few kernel options that *might* be helpful in reducing latency that can be enabled by uncommenting the relevant sections.
There are a couple of lines at the end which disable syncing the system clock to internet time servers. This syncing has in the past been one cause of realtime errors. This might should be made default???
To use, save the script as "rt_setup", make it executable, and run it with sudo because it needs root privileges.Then you'll need to reboot the PC.
One improvement would be to detect dual-core systems or ones without irqbalance and do something different for the IRQ affinity.
****** UPDATE ******
I updated this script and moved it to the first post in this thread.
- Set up isolcpus and a couple of other kernel arguments
- Run update-grub
- Create the IRQ affinity policy script for irqbalance
- Set up irqbalance to use the script.
There are a few kernel options that *might* be helpful in reducing latency that can be enabled by uncommenting the relevant sections.
There are a couple of lines at the end which disable syncing the system clock to internet time servers. This syncing has in the past been one cause of realtime errors. This might should be made default???
To use, save the script as "rt_setup", make it executable, and run it with sudo because it needs root privileges.
sudo ./rt_setup
One improvement would be to detect dual-core systems or ones without irqbalance and do something different for the IRQ affinity.
****** UPDATE ******
I updated this script and moved it to the first post in this thread.
Last edit: 29 Feb 2024 21:11 by mozmck.
The following user(s) said Thank You: seuchato, tommylight, Mecanix
Please Log in or Create an account to join the conversation.
- Ekholm
- Offline
- Senior Member
Less
More
- Posts: 72
- Thank you received: 12
13 Feb 2024 21:13 #293261
by Ekholm
Replied by Ekholm on topic Latency, error finishing read, and IRQ affinity
Attachments:
Please Log in or Create an account to join the conversation.
- Mecanix
- Offline
- Platinum Member
Less
More
- Posts: 446
- Thank you received: 216
13 Feb 2024 21:22 #293264
by Mecanix
Replied by Mecanix on topic Latency, error finishing read, and IRQ affinity
I got the error in the picture. can't find folder
$ sudo apt update && sudo apt upgrade
$ sudo apt install irqbalance
$ sudo ./rt_setup
The following user(s) said Thank You: seuchato
Please Log in or Create an account to join the conversation.
- mozmck
- Offline
- Administrator
Less
More
- Posts: 64
- Thank you received: 55
13 Feb 2024 21:49 - 13 Feb 2024 21:55 #293267
by mozmck
Replied by mozmck on topic Latency, error finishing read, and IRQ affinity
Is irqbalance installed? - Oops, should have refreshed the page before sending. I don't know where else the config file for irqbalance would be than /etc/default
What linux distribution are you running?
What linux distribution are you running?
Last edit: 13 Feb 2024 21:55 by mozmck.
Please Log in or Create an account to join the conversation.
- hmnijp
- Offline
- Junior Member
Less
More
- Posts: 20
- Thank you received: 26
14 Feb 2024 04:36 #293295
by hmnijp
Replied by hmnijp on topic Latency, error finishing read, and IRQ affinity
quotes seem to be missing here? ($2 == 10.10.10.1) →→→ ($2 == "10.10.10.1")Something like this should work:
#!/bin/bash MASK=$( printf "%X" $((1<<`lscpu -e=CPU | tail -1`)) ) NIC=`awk 'BEGIN{nic=""} {if ($1=="iface") {tmpnic = $2} if ($1=="address") {if ($2 == 10.10.10.1) {nic = tmpnic}}} END{print(nic)}' /etc/network/interfaces` if [ "$NIC" != "" ]; then grep $NIC /proc/interrupts | cut -d ":" -f 1 | while read -r IRQ; do echo $MASK > /proc/irq/$IRQ/smp_affinity done fi exit 0"
Please Log in or Create an account to join the conversation.
Time to create page: 0.107 seconds