Debian Stretch 2.7-rtpreempt latency issues.

More
9 years 11 months ago #82991 by jCandlish
Hello Forum

After getting disappointing latency test results from the Jessie uspace buildbot package, I built linuxcnc-2.7.8.7.g8a172f6.1 by the 'apt-get source linuxcnc' method.

I see the same high latencies with my locally built package, and question if latency-test is running with realtime priority. (see attachment).
Code:
latheoperator@125cnc:~$ ps -ef | grep laten latheop+ 4154 3348 0 17:18 pts/0 00:00:00 /bin/bash /usr/bin/latency-test 1000000 1000000 latheop+ 4442 3489 0 17:18 pts/1 00:00:00 grep laten latheoperator@125cnc:~$ chrt -p 4154 pid 4154's current scheduling policy: SCHED_OTHER pid 4154's current scheduling priority: 0 latheoperator@125cnc:~$ pstree 4154 latency-test───halrun───halcmd─┬─pyvcp───{pyvcp} └─rtapi_app───{rtapi_app} latheoperator@125cnc:~$ ps -ef | grep rtapi root 4180 4179 0 17:18 ? 00:00:00 /usr/bin/rtapi_app load threads name1=slow period1=1000000 latheop+ 4630 3489 0 17:19 pts/1 00:00:00 grep rtapi latheoperator@125cnc:~$ chrt -p 4180 pid 4180's current scheduling policy: SCHED_OTHER pid 4180's current scheduling priority: 0 latheoperator@125cnc:~$ ls -l /usr/bin/rtapi_app -rwsr-xr-x 1 root root 102224 Nov 19 15:38 /usr/bin/rtapi_app latheoperator@125cnc:~$ uname -a Linux 125cnc 4.8.0-1-rt-amd64 #1 SMP PREEMPT RT Debian 4.8.5-1 (2016-10-28) x86_64 GNU/Linux

What scheduling policy and priority should latency-test have, and how is this priority established?


Thanks
.

Please Log in to join the conversation.

More
9 years 11 months ago #82998 by PCW
What hardware do you have?

Here are a few latency plots of Preempt-RT

3.18 on a slow machine:

freeby.mesanet.com/atom-330-preempt-rt.png

3.18 on a 3.16 GHz Core Duo

freeby.mesanet.com/e8500-preemt-rt-dc7800-1day.png

3.18 on fairly fast machine

freeby.mesanet.com/h97-g3258-preemt-rt.png

3.18 on a laptop

freeby.mesanet.com/e6420.png

(you cannot change power modes or backlight brightness on this laptop without
large latency spikes, but is fine if you dont mess with those things)

Please Log in to join the conversation.

More
9 years 11 months ago #83000 by jCandlish
The computer is a Shuttle X50V5
Code:
root@125cnc:~# cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 78 model name : Intel(R) Celeron(R) CPU 3855U @ 1.60GHz stepping : 3 microcode : 0x9e cpu MHz : 1600.000 cache size : 2048 KB physical id : 0 siblings : 2 core id : 0 cpu cores : 2 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 22 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave rdrand lahf_lm abm 3dnowprefetch epb intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust erms invpcid rdseed smap clflushopt xsaveopt xsavec xgetbv1 xsaves dtherm arat pln pts hwp hwp_notify hwp_act_window hwp_epp bugs : bogomips : 3216.00 clflush size : 64 cache_alignment : 64 address sizes : 39 bits physical, 48 bits virtual power management: processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 78 model name : Intel(R) Celeron(R) CPU 3855U @ 1.60GHz stepping : 3 microcode : 0x9e cpu MHz : 1600.000 cache size : 2048 KB physical id : 0 siblings : 2 core id : 1 cpu cores : 2 apicid : 2 initial apicid : 2 fpu : yes fpu_exception : yes cpuid level : 22 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave rdrand lahf_lm abm 3dnowprefetch epb intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust erms invpcid rdseed smap clflushopt xsaveopt xsavec xgetbv1 xsaves dtherm arat pln pts hwp hwp_notify hwp_act_window hwp_epp bugs : bogomips : 3217.14 clflush size : 64 cache_alignment : 64 address sizes : 39 bits physical, 48 bits virtual power management: root@125cnc:~# free total used free shared buff/cache available Mem: 3928112 634176 2112840 109388 1181096 3118296 Swap: 6369276 0 6369276 root@125cnc:~# lspci 00:00.0 Host bridge: Intel Corporation Sky Lake Host Bridge/DRAM Registers (rev 08) 00:02.0 VGA compatible controller: Intel Corporation HD Graphics 510 (rev 07) 00:14.0 USB controller: Intel Corporation Sunrise Point-LP USB 3.0 xHCI Controller (rev 21) 00:14.2 Signal processing controller: Intel Corporation Sunrise Point-LP Thermal subsystem (rev 21) 00:16.0 Communication controller: Intel Corporation Sunrise Point-LP CSME HECI (rev 21) 00:17.0 SATA controller: Intel Corporation Sunrise Point-LP SATA Controller [AHCI mode] (rev 21) 00:1d.0 PCI bridge: Intel Corporation Sunrise Point-LP PCI Express Root Port (rev f1) 00:1f.0 ISA bridge: Intel Corporation Sunrise Point-LP LPC Controller (rev 21) 00:1f.2 Memory controller: Intel Corporation Sunrise Point-LP PMC (rev 21) 00:1f.3 Audio device: Intel Corporation Sunrise Point-LP HD Audio (rev 21) 00:1f.4 SMBus: Intel Corporation Sunrise Point-LP SMBus (rev 21) 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection I219-LM (rev 21) 01:00.0 Network controller: Realtek Semiconductor Co., Ltd. RTL8821AE 802.11ac PCIe Wireless Network Adapter

The machine interface is an 7I80HD-16, from order # 2771 on its way from you to me.

I have always used RTAI in the past. My understanding is that the hm2_eth requires an rt-preempt kernel.

I have a little experience with rt-preempt real time audio. There the JACK setup creates an /etc/security/limits.d/audio.conf file, where members of the 'audio' group are granted "rtprio 95" and "memlock unlimited".

I don't see where linuxcnc is setting the realtime parameters for its userspace programs, and checking with 'chrt' seems to indicate latency-test is not running with any elevated real-time privileges.

Please Log in to join the conversation.

More
9 years 11 months ago #83002 by PCW
At least on the latency-histogram, the GUI part does not operate at elevated priority
but must spawn a task that does (if you set the test GUI to RT priority you will get dreadful latency)

Please Log in to join the conversation.

More
9 years 11 months ago #83005 by jCandlish
I don't see any tasks started with elevated priority, or the documentation that explains the proper configuration to achieve such.
Code:
latheoperator@125cnc:~$ pstree 4154 latency-test───halrun───halcmd─┬─pyvcp───{pyvcp} └─rtapi_app───{rtapi_app} latheoperator@125cnc:~$ ps -ef | grep rtapi root 4180 4179 0 17:18 ? 00:00:00 /usr/bin/rtapi_app load threads name1=slow period1=1000000 latheop+ 4630 3489 0 17:19 pts/1 00:00:00 grep rtapi latheoperator@125cnc:~$ chrt -p 4180 pid 4180's current scheduling policy: SCHED_OTHER pid 4180's current scheduling priority: 0

What scheduling policy and priority should rtapi_app have?

Please Log in to join the conversation.

More
9 years 11 months ago #83007 by PCW
If have not seen that it make any difference in the latency (at least with the latency-histogram)

Please Log in to join the conversation.

Time to create page: 0.082 seconds
Powered by Kunena Forum