Hardware latency tests, used PC's
07 Jun 2020 14:10 #170484
by seuchato
Replied by seuchato on topic Hardware latency tests, used PC's
Tommy
I currently have two Dell OptiPlex 980 SFF, i5 650, onboard grafics and LPT. I tested them using this guide: forum.linuxcnc.org/9-installing-linuxcnc...ick-and-easy-install (Excellent work by the way!). Kernel is 4.0.19-0-rt-amd64 I get latency, max jitter of just under 22'000 ns for base thread (25us) thread and just under 20'000 ns for servo thread (1ms) running for about 2 hours with 5 glxgears and watching Balz Müller doing freestyle on vimeo. No Isolcpus, No idle=poll, just bare boot parameters, but in bios all boot parameters set as recommended and pcspkr blacklisted (dunno, if this is really required, but I do not need it anyways). Hope this machine is good for software stepping and/or mesa ethernet control?
One unit is intended for my diy router, the other for a deckel fp2nc. I am looking to get 2 spare 980, one of these will drive a lathe (Schaublin 120VM, retrofitted). If anymore info desired, just let me know.
I could test the following machines:
IBM Z61p, 3 GB Ram, SSD, option: Dock with pciexpress and LPT
Lenovo T500, core2duo, 8 GB ram, option: Dock with pciexpress and LPT
Lenovo W510, i7, 16 GB ram, LPT would be a Exsys EX 1376 on ExpressCard slot
Fujitsu Esprimo 840 (I *think* it is a 840), i7 16 GB ram, guess no LPT possible
HP 8300 Tower, I7, 32 GB ram, on board lpt
Lenovo L540, i5, 16 GB Ram, LPT would be a Exsys EX 1376 on ExpressCard slot
Lenovo w541, i7, 32 GB Ram, LPT would be a Exsys EX 1376 on ExpressCard slot
Lenovo p50, i7, 32 GB ram, LPT would be a Exsys EX 1376 on ExpressCard slot
Lenovo T460p, i7, 32 GB ram, LPT would be a Exsys EX 1376 on ExpressCard slot
Any interest?
I think, wiki.linuxcnc.org/cgi-bin/wiki.pl?Latency-Test is a bit outdated and not as complete as it could be. How about a new page with more up to date info?
By the way: Tommy, how did you test? I guess with sata disc exchange?
Greez
chris
I currently have two Dell OptiPlex 980 SFF, i5 650, onboard grafics and LPT. I tested them using this guide: forum.linuxcnc.org/9-installing-linuxcnc...ick-and-easy-install (Excellent work by the way!). Kernel is 4.0.19-0-rt-amd64 I get latency, max jitter of just under 22'000 ns for base thread (25us) thread and just under 20'000 ns for servo thread (1ms) running for about 2 hours with 5 glxgears and watching Balz Müller doing freestyle on vimeo. No Isolcpus, No idle=poll, just bare boot parameters, but in bios all boot parameters set as recommended and pcspkr blacklisted (dunno, if this is really required, but I do not need it anyways). Hope this machine is good for software stepping and/or mesa ethernet control?
One unit is intended for my diy router, the other for a deckel fp2nc. I am looking to get 2 spare 980, one of these will drive a lathe (Schaublin 120VM, retrofitted). If anymore info desired, just let me know.
I could test the following machines:
IBM Z61p, 3 GB Ram, SSD, option: Dock with pciexpress and LPT
Lenovo T500, core2duo, 8 GB ram, option: Dock with pciexpress and LPT
Lenovo W510, i7, 16 GB ram, LPT would be a Exsys EX 1376 on ExpressCard slot
Fujitsu Esprimo 840 (I *think* it is a 840), i7 16 GB ram, guess no LPT possible
HP 8300 Tower, I7, 32 GB ram, on board lpt
Lenovo L540, i5, 16 GB Ram, LPT would be a Exsys EX 1376 on ExpressCard slot
Lenovo w541, i7, 32 GB Ram, LPT would be a Exsys EX 1376 on ExpressCard slot
Lenovo p50, i7, 32 GB ram, LPT would be a Exsys EX 1376 on ExpressCard slot
Lenovo T460p, i7, 32 GB ram, LPT would be a Exsys EX 1376 on ExpressCard slot
Any interest?
I think, wiki.linuxcnc.org/cgi-bin/wiki.pl?Latency-Test is a bit outdated and not as complete as it could be. How about a new page with more up to date info?
By the way: Tommy, how did you test? I guess with sata disc exchange?
Greez
chris
The following user(s) said Thank You: tommylight
Please Log in or Create an account to join the conversation.
- tommylight
- Topic Author
- Away
- Moderator
Less
More
- Posts: 19188
- Thank you received: 6430
07 Jun 2020 15:09 - 07 Jun 2020 16:13 #170491
by tommylight
Replied by tommylight on topic Hardware latency tests, used PC's
That is very good latency, and very usable for software stepping.
From that list, a guess would be:
Z61p might be good, needs disabling everything in BIOS,
Fujitsu might be good, most of Fujitsu's i have tested lately are very good, even laptops,
HP8300 is the preferred machine for Eth by PCW,
Lenovo's, there might be a surprise in that list, but X220 and t420s are not usable for machine control at all.
If you have access to those, please do some tests, use a USB and one of the ready made images, or an SSD with Linuxcnc installed.
You can install Linux and Linuxcnc on a PC, yank it out, put a USB to SATA adapter and plug it to any PC or Laptop and it will boot and work properly ! Make windows do that, not in your or my lifetime !
Usually i mention if it was booted from USB or installed on HDD or SSD as those do make some difference, mainly the latency will be lower when booted from USB stick or SSD than when installed to a HDD.
Thank you very much for the info.
From that list, a guess would be:
Z61p might be good, needs disabling everything in BIOS,
Fujitsu might be good, most of Fujitsu's i have tested lately are very good, even laptops,
HP8300 is the preferred machine for Eth by PCW,
Lenovo's, there might be a surprise in that list, but X220 and t420s are not usable for machine control at all.
If you have access to those, please do some tests, use a USB and one of the ready made images, or an SSD with Linuxcnc installed.
You can install Linux and Linuxcnc on a PC, yank it out, put a USB to SATA adapter and plug it to any PC or Laptop and it will boot and work properly ! Make windows do that, not in your or my lifetime !
Usually i mention if it was booted from USB or installed on HDD or SSD as those do make some difference, mainly the latency will be lower when booted from USB stick or SSD than when installed to a HDD.
Thank you very much for the info.
Last edit: 07 Jun 2020 16:13 by tommylight. Reason: typo
Please Log in or Create an account to join the conversation.
- tommylight
- Topic Author
- Away
- Moderator
Less
More
- Posts: 19188
- Thank you received: 6430
07 Jun 2020 16:12 - 07 Jun 2020 16:39 #170501
by tommylight
Replied by tommylight on topic Hardware latency tests, used PC's
One important thing to keep in mind during latency testing, resize the glxgears and youtube videos to full screen and back, as that will cause spikes always, especially the 4k videos on older PC's.
Command to use for latency testing in terminal:For base and servo thread:
Command to use for latency testing in terminal:
latency-histogram --nobase --sbinsize 1000 --show
latency-histogram --bbinsize 1000 --sbinsize 1000 --show
Last edit: 07 Jun 2020 16:39 by tommylight. Reason: More info
The following user(s) said Thank You: seuchato
Please Log in or Create an account to join the conversation.
- tommylight
- Topic Author
- Away
- Moderator
Less
More
- Posts: 19188
- Thank you received: 6430
07 Jun 2020 16:29 #170502
by tommylight
Was going through this topic and adding additional info, so this made me laugh again.
Thank you Beagle.
Plenty of added info to the original post and the subsequent ones, so if interested, please do have a go around again.
Thank you, all.
Replied by tommylight on topic Hardware latency tests, used PC's
LOL, again.24hrs to compile a kernel ?
Are you using pen & paper, I thought 2 hours was slow.
Was going through this topic and adding additional info, so this made me laugh again.
Thank you Beagle.
Plenty of added info to the original post and the subsequent ones, so if interested, please do have a go around again.
Thank you, all.
Please Log in or Create an account to join the conversation.
11 Jun 2020 09:00 #171097
by seuchato
You're running through wide open doors: I have two laptops running windoze as second os, the rest of my machines are all only linux.
Replied by seuchato on topic Hardware latency tests, used PC's
...
You can install Linux and Linuxcnc on a PC, yank it out, put a USB to SATA adapter and plug it to any PC or Laptop and it will boot and work properly ! Make windows do that, not in your or my lifetime !
...
You're running through wide open doors: I have two laptops running windoze as second os, the rest of my machines are all only linux.
The following user(s) said Thank You: tommylight
Please Log in or Create an account to join the conversation.
11 Jun 2020 09:04 #171098
by seuchato
I am writing a script to test my machines. If it does what I want, I will put it here in a new thread. I will include your suggestion for maximising glxgears and video windows and the base and servo thread command. You happen to know how to send a keystroke to an app in bash? Cause that's what I would love to do from time after time in my bash script.
greez
chris
Replied by seuchato on topic Hardware latency tests, used PC's
One important thing to keep in mind during latency testing, resize the glxgears and youtube videos to full screen and back, as that will cause spikes always, especially the 4k videos on older PC's.
Command to use for latency testing in terminal:For base and servo thread:latency-histogram --nobase --sbinsize 1000 --showlatency-histogram --bbinsize 1000 --sbinsize 1000 --show
I am writing a script to test my machines. If it does what I want, I will put it here in a new thread. I will include your suggestion for maximising glxgears and video windows and the base and servo thread command. You happen to know how to send a keystroke to an app in bash? Cause that's what I would love to do from time after time in my bash script.
greez
chris
The following user(s) said Thank You: tommylight
Please Log in or Create an account to join the conversation.
- tommylight
- Topic Author
- Away
- Moderator
Less
More
- Posts: 19188
- Thank you received: 6430
11 Jun 2020 11:30 #171122
by tommylight
Replied by tommylight on topic Hardware latency tests, used PC's
Nope. Sorry.You happen to know how to send a keystroke to an app in bash? Cause that's what I would love to do from time after time in my bash script.
Please Log in or Create an account to join the conversation.
15 Jun 2020 08:50 #171711
by seuchato
Replied by seuchato on topic Hardware latency tests, used PC's
I got a script up and am test running for 2 hours now. If it succeeds, I will publish it as a proposal and subject to improvement.
Two questions:
chris
Two questions:
- What is a reasonable time for "latency-histogram --bbinsize 1000 --sbinsize 1000 --show" to run? Is two hours enough?
- I remember a PC having unexpected RT problems. The machine is at friends house now. He will be willing to let me test ist. How do I capture the expected RT-Errors?
chris
The following user(s) said Thank You: tommylight
Please Log in or Create an account to join the conversation.
- tommylight
- Topic Author
- Away
- Moderator
Less
More
- Posts: 19188
- Thank you received: 6430
15 Jun 2020 09:44 #171723
by tommylight
Replied by tommylight on topic Hardware latency tests, used PC's
There is one issue to be aware of when testing for latency on 64 bit machines with "uspace" flavour of Linuxcnc:
-Wen running latency histogram, it will set the processor at max speed and will not allow them to go down as long as the test is running,
-When running LinuxCNC on the same machine the processors will throttle up and down depending on load,
That leads to the following:
-While running LinuxCNC there is still the possibility of latency excursions despite getting good latency numbers on latency histogram,
-Setting the CPU governor to max performance should avoid this issue
-On low quality laptops setting the processor to max performance will cause them to overheat, and all laptops will run at high processor temperature making the possibility of thermal throttling very high, hence risk of latency excursions,
-LinuxCNC runs nicely on very few laptops without setting the processor to max performance and the temperatures remaining at a reasonable levels,
-This is not a big issue for PC's as most of them can handle higher temperatures easily,
-Some laptops have BIOS settings to set the processor at the lowest frequency and disable speedstep during work, this can work nicely for LinuxCNC despite low frequency.
-
-Running latency tests for an hour should be enough to get a reasonable result to grant real life tests,
-Real life tests should be run for at least a day with Mesa board active, stepgens are good for this as the tests can be done without any hardware attached, i test them for two days, so 48 hours of running gcode without dropping the connection or latency errors.
-
Latency errors for software stepping using base period/base thread can be easily avoided by setting the base thread to a higher value within reason, on some machines that do not require high frequency stepping i can set the base thread to 300000 and have a working machine despite bad latency,
Similarly, setting the servo thread to 2000000 can make some PC's/laptops usable, although i never use this as the risk of errors is to high.
-Wen running latency histogram, it will set the processor at max speed and will not allow them to go down as long as the test is running,
-When running LinuxCNC on the same machine the processors will throttle up and down depending on load,
That leads to the following:
-While running LinuxCNC there is still the possibility of latency excursions despite getting good latency numbers on latency histogram,
-Setting the CPU governor to max performance should avoid this issue
-On low quality laptops setting the processor to max performance will cause them to overheat, and all laptops will run at high processor temperature making the possibility of thermal throttling very high, hence risk of latency excursions,
-LinuxCNC runs nicely on very few laptops without setting the processor to max performance and the temperatures remaining at a reasonable levels,
-This is not a big issue for PC's as most of them can handle higher temperatures easily,
-Some laptops have BIOS settings to set the processor at the lowest frequency and disable speedstep during work, this can work nicely for LinuxCNC despite low frequency.
-
-Running latency tests for an hour should be enough to get a reasonable result to grant real life tests,
-Real life tests should be run for at least a day with Mesa board active, stepgens are good for this as the tests can be done without any hardware attached, i test them for two days, so 48 hours of running gcode without dropping the connection or latency errors.
-
Latency errors for software stepping using base period/base thread can be easily avoided by setting the base thread to a higher value within reason, on some machines that do not require high frequency stepping i can set the base thread to 300000 and have a working machine despite bad latency,
Similarly, setting the servo thread to 2000000 can make some PC's/laptops usable, although i never use this as the risk of errors is to high.
Please Log in or Create an account to join the conversation.
15 Jun 2020 11:30 #171739
by seuchato
Replied by seuchato on topic Hardware latency tests, used PC's
tommylight, Thanks a ton for your valuable explanations. Things up in clouds start up to get clearer thanks to your explanation
So, if I get you right, we should set CPU governor to performance for real life too?
Could be done just before starting linuxcnc like this I guess:
Regarding my test script: ran through for two hours now. It provides a set of info on the machine, a nice histogram. I still don't like to have to manually hit the screenshot button to safe the histogram. I know, I can run latency-histogram with "--nox" but then, I do not have the graphical histogram and a huge data grave. Is there a way to get the data the graphical histogram collects, summed up in a text file?
Plus: got the machine from my friend, provides reliable "realtime delay" errors. Still struggling to get these Errors logged in a file.
greez
chris
So, if I get you right, we should set CPU governor to performance for real life too?
Could be done just before starting linuxcnc like this I guess:
sudo apt-get install cpufrequtils
echo 'GOVERNOR="performance"' | sudo tee /etc/default/cpufrequtils
sudo systemctl disable ondemand
Regarding my test script: ran through for two hours now. It provides a set of info on the machine, a nice histogram. I still don't like to have to manually hit the screenshot button to safe the histogram. I know, I can run latency-histogram with "--nox" but then, I do not have the graphical histogram and a huge data grave. Is there a way to get the data the graphical histogram collects, summed up in a text file?
Plus: got the machine from my friend, provides reliable "realtime delay" errors. Still struggling to get these Errors logged in a file.
greez
chris
The following user(s) said Thank You: tommylight
Please Log in or Create an account to join the conversation.
Time to create page: 0.115 seconds