Realtime delay error immediately after opening Linuxcnc
- carson
- Offline
- New Member
Less
More
- Posts: 13
- Thank you received: 2
15 Mar 2019 03:04 #128630
by carson
Realtime delay error immediately after opening Linuxcnc was created by carson
I just did a reinstall of linuxcnc using the stock Debian Wheezy with linuxcnc 2.7. I was running Ubuntu 12.04 with linuxcnc 2.5 on the same computer and it was working pretty well. I was getting maximum base thread jitters around 15 us. I would occasionally get the error: "RTAPI: ERROR: Unexpected realtime delay on task 1" but only after the monitor had gone to sleep for a while. With the new install I am getting max base thread jitters around 25 us but I get the error "RTAPI: ERROR: Unexpected realtime delay on task 1" immediately upon opening linuxcnc. I was getting much larger jitters before disabling hardware acceleration on the graphics card driver. So I'm still suspicious that the graphics driver might be the problem. I don't think it's any BIOS settings since I haven't changed those and the same hardware was working with the previous version. It's a quad core and I tried adding isolcpus=2,3 to the boot config but that didn't help.
Is there anyway to diagnose the source of that realtime delay error? Why would this be happening if my maximum jitter is 25 us (which is fast enough according to the documentation)? What actually triggers this error? Dmesg doesn't seem to have any useful info, just the error.
Thanks for any help,
Carson
Is there anyway to diagnose the source of that realtime delay error? Why would this be happening if my maximum jitter is 25 us (which is fast enough according to the documentation)? What actually triggers this error? Dmesg doesn't seem to have any useful info, just the error.
Thanks for any help,
Carson
Please Log in or Create an account to join the conversation.
- pl7i92
- Offline
- Platinum Member
Less
More
- Posts: 1875
- Thank you received: 355
15 Mar 2019 07:47 #128639
by pl7i92
Replied by pl7i92 on topic Realtime delay error immediately after opening Linuxcnc
you may check the latency again
wiki.linuxcnc.org/cgi-bin/wiki.pl?Latency-Test
and take action on smi
also try to take out Bios Treads Energie
and Sound
wiki.linuxcnc.org/cgi-bin/wiki.pl?FixingSMIIssues
wiki.linuxcnc.org/cgi-bin/wiki.pl?Latency-Test
and take action on smi
also try to take out Bios Treads Energie
and Sound
wiki.linuxcnc.org/cgi-bin/wiki.pl?FixingSMIIssues
Please Log in or Create an account to join the conversation.
- Richard J Kinch
- Offline
- Senior Member
Less
More
- Posts: 61
- Thank you received: 4
15 Mar 2019 08:00 #128643
by Richard J Kinch
If the jitter has gotten worse (15 us before, now 25 us), then have you adjusted the base thread period in the .ini file accordingly?
The "fast enough" criterion just means the system is a candidate for performing a decent base thread period. What period is achievable is constrained by the jitter, so you have to tune the period down (to be longer, slower) if the jitter has gotten worse. That slower period should still be decent, just less decent than before, if you will.
Replied by Richard J Kinch on topic Realtime delay error immediately after opening Linuxcnc
I was getting maximum base thread jitters around 15 us. ... Why would this be happening if my maximum jitter is 25 us (which is fast enough according to the documentation)?
If the jitter has gotten worse (15 us before, now 25 us), then have you adjusted the base thread period in the .ini file accordingly?
The "fast enough" criterion just means the system is a candidate for performing a decent base thread period. What period is achievable is constrained by the jitter, so you have to tune the period down (to be longer, slower) if the jitter has gotten worse. That slower period should still be decent, just less decent than before, if you will.
Please Log in or Create an account to join the conversation.
- carson
- Offline
- New Member
Less
More
- Posts: 13
- Thank you received: 2
16 Mar 2019 16:07 #128746
by carson
Replied by carson on topic Realtime delay error immediately after opening Linuxcnc
Thanks for the responses.
pl7i92: I have run the latency test many times for at least an hour while trying to stress the system by running glxgears, GIMP, VLC, surfing the web, etc. The base thread jitter never gets above 26 us. I never see any periodic spikes as described for SMI issues and I have an AMD chipset, not Intel.
Richard: I did adjust the base_thread parameter in my .ini file to 30 us. However, I just tried increasing all the way to 100 us and in that case, the unexpected realtime delay error does not immediately appear. But it pops up if I start running glxgears or start drawing something in GIMP or open VLC. Is that what determines when this error is thrown: whenever linuxcnc detects a latency larger than the base_thread parameter?
It seems like the results of the latency test are somehow inaccurate or not representative of latencies seen by linuxcnc.
I'm wondering if I can just ignore this error since the latency test seems to indicate that things are working fine.
pl7i92: I have run the latency test many times for at least an hour while trying to stress the system by running glxgears, GIMP, VLC, surfing the web, etc. The base thread jitter never gets above 26 us. I never see any periodic spikes as described for SMI issues and I have an AMD chipset, not Intel.
Richard: I did adjust the base_thread parameter in my .ini file to 30 us. However, I just tried increasing all the way to 100 us and in that case, the unexpected realtime delay error does not immediately appear. But it pops up if I start running glxgears or start drawing something in GIMP or open VLC. Is that what determines when this error is thrown: whenever linuxcnc detects a latency larger than the base_thread parameter?
It seems like the results of the latency test are somehow inaccurate or not representative of latencies seen by linuxcnc.
I'm wondering if I can just ignore this error since the latency test seems to indicate that things are working fine.
Please Log in or Create an account to join the conversation.
- tommylight
- Away
- Moderator
Less
More
- Posts: 19533
- Thank you received: 6555
16 Mar 2019 16:32 #128751
by tommylight
Replied by tommylight on topic Realtime delay error immediately after opening Linuxcnc
You have some power saving options active in bios or you have an Nvidia graphic card.
Disable everything related to power saving, C states, virtualisation etc in the BIOS and try again.
Disable everything related to power saving, C states, virtualisation etc in the BIOS and try again.
Please Log in or Create an account to join the conversation.
- carson
- Offline
- New Member
Less
More
- Posts: 13
- Thank you received: 2
16 Mar 2019 23:16 #128781
by carson
Replied by carson on topic Realtime delay error immediately after opening Linuxcnc
Thanks Tommy,
I cured the problem by disabling APIC in the BIOS. I guess that has been suggested on other threads but I just thought that since I hadn't changed the hardware, that wouldn't be an issue.
I still don't understand why that setting is an issue for the updated system (with Debian Wheezy and Linuxcnc 2.7) but not for the older OS with Linuxcnc 2.5.
And I still feel like there was a discrepancy between the latency test and the realtime error thrown in Linuxcnc. But oh well, I guess it's working now.
One more question: I tried to add the kernel parameter "noapic" instead of disabling it in the bios, but that didn't seem to work (the realtime error only goes away when I disable apic in the bios). Does anyone know what the difference is? I dual boot so it would be nice to have apic enabled for the other OS since things seem slower without it.
I cured the problem by disabling APIC in the BIOS. I guess that has been suggested on other threads but I just thought that since I hadn't changed the hardware, that wouldn't be an issue.
I still don't understand why that setting is an issue for the updated system (with Debian Wheezy and Linuxcnc 2.7) but not for the older OS with Linuxcnc 2.5.
And I still feel like there was a discrepancy between the latency test and the realtime error thrown in Linuxcnc. But oh well, I guess it's working now.
One more question: I tried to add the kernel parameter "noapic" instead of disabling it in the bios, but that didn't seem to work (the realtime error only goes away when I disable apic in the bios). Does anyone know what the difference is? I dual boot so it would be nice to have apic enabled for the other OS since things seem slower without it.
Please Log in or Create an account to join the conversation.
- tommylight
- Away
- Moderator
Less
More
- Posts: 19533
- Thank you received: 6555
16 Mar 2019 23:47 #128785
by tommylight
Replied by tommylight on topic Realtime delay error immediately after opening Linuxcnc
Did you try "lapic" ?
I remember that from the days of Linuxcnc when it was EMC2 on Ubuntu 8.04, had to add that for older laptops.
Not sure if that would help, and not sure that disabling that will have such an impact on computer speed. It is used for the ability to use more than 15 IRQ ( interrupt requests ) but i have never seen anything use IRQ's over number 15, ever.
I remember that from the days of Linuxcnc when it was EMC2 on Ubuntu 8.04, had to add that for older laptops.
Not sure if that would help, and not sure that disabling that will have such an impact on computer speed. It is used for the ability to use more than 15 IRQ ( interrupt requests ) but i have never seen anything use IRQ's over number 15, ever.
Please Log in or Create an account to join the conversation.
- Richard J Kinch
- Offline
- Senior Member
Less
More
- Posts: 61
- Thank you received: 4
17 Mar 2019 10:14 - 17 Mar 2019 10:19 #128809
by Richard J Kinch
Yes, for some proportion of "larger". Theoretical realtime operation must presume some upper bound on latency to be able to work. It is able to detect violations as a logical event, but it can't recover the information lost in an excessive delay, or thus know the magnitude of failure in the planned motion.
Correct, the latency testing is essentially trying to prove a negative, which is not completely reliable, especially since PCs are so heavily larded with anti-realtime BIOS gimmicks nowadays, that can't be fully disabled.
You can ignore it if you have some way to keep watch on the effect.
For some years my milling machine had occasional realtime delays that I could not track down. It could run for hours without experiencing them, but once in a while it would occur. But I was able to measure the lost steps during a session by zeroing the hand crank dials to the home position. That way I started the machine at the home position, and shut it down by returning to home, and the dials would then represent the sum of the lost steps during the work session. If you had independent DRO scales, that would perform such monitoring more precisely.
My solution to these mysteries was to standardize on the ASRock Q1900B-ITX motherboard, which my experience has proven to be the least corrupt with realtime surprises.
Replied by Richard J Kinch on topic Realtime delay error immediately after opening Linuxcnc
Is that what determines when this error is thrown: whenever linuxcnc detects a latency larger than the base_thread parameter?
Yes, for some proportion of "larger". Theoretical realtime operation must presume some upper bound on latency to be able to work. It is able to detect violations as a logical event, but it can't recover the information lost in an excessive delay, or thus know the magnitude of failure in the planned motion.
It seems like the results of the latency test are somehow inaccurate or not representative of latencies seen by linuxcnc.
Correct, the latency testing is essentially trying to prove a negative, which is not completely reliable, especially since PCs are so heavily larded with anti-realtime BIOS gimmicks nowadays, that can't be fully disabled.
I'm wondering if I can just ignore this error since the latency test seems to indicate that things are working fine.
You can ignore it if you have some way to keep watch on the effect.
For some years my milling machine had occasional realtime delays that I could not track down. It could run for hours without experiencing them, but once in a while it would occur. But I was able to measure the lost steps during a session by zeroing the hand crank dials to the home position. That way I started the machine at the home position, and shut it down by returning to home, and the dials would then represent the sum of the lost steps during the work session. If you had independent DRO scales, that would perform such monitoring more precisely.
My solution to these mysteries was to standardize on the ASRock Q1900B-ITX motherboard, which my experience has proven to be the least corrupt with realtime surprises.
Last edit: 17 Mar 2019 10:19 by Richard J Kinch.
Please Log in or Create an account to join the conversation.
Time to create page: 0.062 seconds