Realtime delay error immediately after opening Linuxcnc

More
15 Mar 2019 03:04 #128630 by carson
I just did a reinstall of linuxcnc using the stock Debian Wheezy with linuxcnc 2.7. I was running Ubuntu 12.04 with linuxcnc 2.5 on the same computer and it was working pretty well. I was getting maximum base thread jitters around 15 us. I would occasionally get the error: "RTAPI: ERROR: Unexpected realtime delay on task 1" but only after the monitor had gone to sleep for a while. With the new install I am getting max base thread jitters around 25 us but I get the error "RTAPI: ERROR: Unexpected realtime delay on task 1" immediately upon opening linuxcnc. I was getting much larger jitters before disabling hardware acceleration on the graphics card driver. So I'm still suspicious that the graphics driver might be the problem. I don't think it's any BIOS settings since I haven't changed those and the same hardware was working with the previous version. It's a quad core and I tried adding isolcpus=2,3 to the boot config but that didn't help.

Is there anyway to diagnose the source of that realtime delay error? Why would this be happening if my maximum jitter is 25 us (which is fast enough according to the documentation)? What actually triggers this error? Dmesg doesn't seem to have any useful info, just the error.

Thanks for any help,
Carson

Please Log in or Create an account to join the conversation.

More
15 Mar 2019 07:47 #128639 by pl7i92
you may check the latency again
wiki.linuxcnc.org/cgi-bin/wiki.pl?Latency-Test

and take action on smi
also try to take out Bios Treads Energie
and Sound
wiki.linuxcnc.org/cgi-bin/wiki.pl?FixingSMIIssues

Please Log in or Create an account to join the conversation.

More
15 Mar 2019 08:00 #128643 by Richard J Kinch

carson wrote: I was getting maximum base thread jitters around 15 us. ... Why would this be happening if my maximum jitter is 25 us (which is fast enough according to the documentation)?


If the jitter has gotten worse (15 us before, now 25 us), then have you adjusted the base thread period in the .ini file accordingly?

The "fast enough" criterion just means the system is a candidate for performing a decent base thread period. What period is achievable is constrained by the jitter, so you have to tune the period down (to be longer, slower) if the jitter has gotten worse. That slower period should still be decent, just less decent than before, if you will.

Please Log in or Create an account to join the conversation.

More
16 Mar 2019 16:07 #128746 by carson
Thanks for the responses.
pl7i92: I have run the latency test many times for at least an hour while trying to stress the system by running glxgears, GIMP, VLC, surfing the web, etc. The base thread jitter never gets above 26 us. I never see any periodic spikes as described for SMI issues and I have an AMD chipset, not Intel.

Richard: I did adjust the base_thread parameter in my .ini file to 30 us. However, I just tried increasing all the way to 100 us and in that case, the unexpected realtime delay error does not immediately appear. But it pops up if I start running glxgears or start drawing something in GIMP or open VLC. Is that what determines when this error is thrown: whenever linuxcnc detects a latency larger than the base_thread parameter?
It seems like the results of the latency test are somehow inaccurate or not representative of latencies seen by linuxcnc.
I'm wondering if I can just ignore this error since the latency test seems to indicate that things are working fine.

Please Log in or Create an account to join the conversation.

More
16 Mar 2019 16:32 #128751 by tommylight
You have some power saving options active in bios or you have an Nvidia graphic card.
Disable everything related to power saving, C states, virtualisation etc in the BIOS and try again.

Please Log in or Create an account to join the conversation.

More
16 Mar 2019 23:16 #128781 by carson
Thanks Tommy,
I cured the problem by disabling APIC in the BIOS. I guess that has been suggested on other threads but I just thought that since I hadn't changed the hardware, that wouldn't be an issue.
I still don't understand why that setting is an issue for the updated system (with Debian Wheezy and Linuxcnc 2.7) but not for the older OS with Linuxcnc 2.5.
And I still feel like there was a discrepancy between the latency test and the realtime error thrown in Linuxcnc. But oh well, I guess it's working now.

One more question: I tried to add the kernel parameter "noapic" instead of disabling it in the bios, but that didn't seem to work (the realtime error only goes away when I disable apic in the bios). Does anyone know what the difference is? I dual boot so it would be nice to have apic enabled for the other OS since things seem slower without it.

Please Log in or Create an account to join the conversation.

More
16 Mar 2019 23:47 #128785 by tommylight
Did you try "lapic" ?
I remember that from the days of Linuxcnc when it was EMC2 on Ubuntu 8.04, had to add that for older laptops.
Not sure if that would help, and not sure that disabling that will have such an impact on computer speed. It is used for the ability to use more than 15 IRQ ( interrupt requests ) but i have never seen anything use IRQ's over number 15, ever.

Please Log in or Create an account to join the conversation.

More
17 Mar 2019 10:14 - 17 Mar 2019 10:19 #128809 by Richard J Kinch

carson wrote: Is that what determines when this error is thrown: whenever linuxcnc detects a latency larger than the base_thread parameter?


Yes, for some proportion of "larger". Theoretical realtime operation must presume some upper bound on latency to be able to work. It is able to detect violations as a logical event, but it can't recover the information lost in an excessive delay, or thus know the magnitude of failure in the planned motion.

It seems like the results of the latency test are somehow inaccurate or not representative of latencies seen by linuxcnc.


Correct, the latency testing is essentially trying to prove a negative, which is not completely reliable, especially since PCs are so heavily larded with anti-realtime BIOS gimmicks nowadays, that can't be fully disabled.

I'm wondering if I can just ignore this error since the latency test seems to indicate that things are working fine.


You can ignore it if you have some way to keep watch on the effect.

For some years my milling machine had occasional realtime delays that I could not track down. It could run for hours without experiencing them, but once in a while it would occur. But I was able to measure the lost steps during a session by zeroing the hand crank dials to the home position. That way I started the machine at the home position, and shut it down by returning to home, and the dials would then represent the sum of the lost steps during the work session. If you had independent DRO scales, that would perform such monitoring more precisely.

My solution to these mysteries was to standardize on the ASRock Q1900B-ITX motherboard, which my experience has proven to be the least corrupt with realtime surprises.
Last edit: 17 Mar 2019 10:19 by Richard J Kinch.

Please Log in or Create an account to join the conversation.

Time to create page: 0.087 seconds
Powered by Kunena Forum