Problems with sporadic freezes and high jitter

More
31 Oct 2013 14:33 #40437 by maexbecks
Hi,
I use the Nexcom NISE 2010 to control my machines. It works fine generally, nevertheless it has some problems with Ubuntu 8.04 and 10.04.
With Ubuntu 8.04 installed from the LinuxCNC LiveCD I get a very high jitter (200.000 ns). I tried everything from the wiki to lower the jitter, but nothing worked, also because Nexcom managed to remove every mentioned option from the BIOS. Anyway the machines work with this version, but I want them to work perfectly.
With Ubuntu 10.04 installed from the LinuxCNC LiveCD aswell I have a low jitter (15.000 - 20.000 ns), but the Computer freezes sporadicly. The whole system freezes so that I can not even switch to the terminal with ALT + CTRL + F1.

What may be different from 10.04 to 8.04 so that the jitter is lower? What could be the problem with the freezes? Are there some log files I can look at? Maybe someone uses the same Board and managed to keep the jitter low on 8.04?

Please Log in or Create an account to join the conversation.

More
31 Oct 2013 19:46 #40449 by ArcEye
Hi

The Intel Atom boards most used, have a dual core processor. So far as I can tell this is a single core and thus it should work better with the non-SMP kernel such as the one in the 8.04 based distro.
200K is very high and completely useless for stepper systems and probably no use for hardware pulses or servos either.

What you probably need to do with 8.04 is run the rtai latency test as described in www.linuxcnc.org/index.php/english/forum...-the-latency-problem

With that running in a terminal you should be able to narrow down what is causing the spikes (200K is too high to be a consistant average, there must be spikes in there).

Sometimes a USB mouse as opposed to a PS2 one can have effects and vice versa. The other favorite is graphics, does tabbing between windows, moving windows around etc forcing redraws, trigger the spikes.
Starting Linuxcnc with Axis as the GUI triggers a heavy openGL graphics load. Starting Firefox from cold, has a graphics, network and disk access load.

That 10.04 gives much better latency could be down to graphics driver versions, if it was a quite new chipset, it might not be properly supported by the much earlier kernel with 8.04
Do a lspci -vv > lspci.txt and post it. Look at what driver is loaded for the graphics and chase down what version 10.04 uses, if they are the same name ( info will be in the respective xserver-xorg-video-DRIVERNAME.deb )

Regards the lockups with 10.04, need to know when exactly they occur, what you were doing at the time.
If you leave it running with nothing loaded does it freeze? Normally not, there has to be a trigger.
Again the freeze you describe sounds as though it could be graphics related.

Do a dmesg > dmesg.txt and also attach /var/log/Xorg.0.log and possibly /var/log/kern.log

The problem with these lock-ups is that largely they do not leave a good trace, because all logging stops and the user has to crash out.

If you can provide some more detail, we might be able to narrow down the problem and suggest remedies

regards
The following user(s) said Thank You: maexbecks

Please Log in or Create an account to join the conversation.

More
01 Nov 2013 00:23 - 01 Nov 2013 00:24 #40461 by maexbecks
While running the latency test with 8.04 you can see that the average jitter is about 5000 ns, but every 2-3 seconds the spikes appear. With/out USB mouse and PS2 keyboard connected the result is the same. So it must be some periodical interrupt. I turned SMI off aswell.

Tomorrow I will install 10.04 on a new NISE2010 and upload both lspci outputs. But I couldn't find any graphics driver information in the lspci output of 8.04. Could it be that the graphics controller is not used at all with 8.04 and everything is calculated by the CPU?

The lockups with 10.04 seem to be randomly. I only saw the lockup once and at this point I closed LinuxCNC, opened the thunar filemanagement and while I was moving the mouse cursor to a file in thunar everything froze and only switching the computer off manually helped. When I install 10.04 on the new computer tomorrow I will make a long time test, to see if the system locks up when nothing happens.

So far thank you very much for your help.
Last edit: 01 Nov 2013 00:24 by maexbecks.

Please Log in or Create an account to join the conversation.

More
01 Nov 2013 00:48 #40462 by ArcEye

So it must be some periodical interrupt. I turned SMI off aswell.


SMI is normally every 32 or 64 seconds

The spec says Chipset IntelĀ® ICH7M which is ICH7 revision 13 or 14 depending upon whether they use the letter I

My FAQ at www.linuxcnc.org/index.php/english/forum...me-latency-solutions
lists the chipsets which the SMI fix covers, and it only covers revision 0 and 1 of the ICH7 range

If you load the smi module and it finds a chipset it does not cover, it will just unload again (do a lsmod in another terminal whilst linuxcnc is running)

But I couldn't find any graphics driver information in the lspci output of 8.04. Could it be that the graphics controller is not used at all with 8.04 and everything is calculated by the CPU?


It is normally the second entry, after Host Bridge and starts VGA Compatible Controller:
I have never seen a print that did not contain it.
If info is lacking run sudo lspci -vv | more which will prevent any access denied entries and show it to you one page at a time

regards
The following user(s) said Thank You: maexbecks

Please Log in or Create an account to join the conversation.

More
01 Nov 2013 19:10 - 01 Nov 2013 20:02 #40483 by maexbecks

If you load the smi module and it finds a chipset it does not cover, it will just unload again (do a lsmod in another terminal whilst linuxcnc is running)


lsmod says
rtai_smi     3600   0
So I guess it is loaded, but it's not used by anything. Or does this mean, that it is not loaded?

I attached the lspci output of 8.04. There is the VGA Compatible Controller.
00:02.0 VGA compatible controller: Intel Corporation Mobile 945GME Express Integrated Graphics Controller (rev 03) (prog-if 00 [VGA controller])
	Subsystem: Intel Corporation Mobile 945GME Express Integrated Graphics Controller
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR-
	Interrupt: pin A routed to IRQ 10
	Region 0: Memory at fdf00000 (32-bit, non-prefetchable) [size=512K]
	Region 1: I/O ports at ff00 [size=8]
	Region 2: Memory at d0000000 (32-bit, prefetchable) [size=256M]
	Region 3: Memory at fdf80000 (32-bit, non-prefetchable) [size=256K]
	Capabilities: <access denied>

Edit: Now I also attached the lspci output of 10.04. There you can clearly see the driver i915, which you cannot in the other. Maybe this is the point?
Attachments:
Last edit: 01 Nov 2013 20:02 by maexbecks.

Please Log in or Create an account to join the conversation.

More
01 Nov 2013 23:11 #40491 by ArcEye
Hi

You may be on to something with the i915 driver
That is a DRI (direct rendering interface) driver and also modesetting so does not require fbdev loaded as well

The i915 driver is now almost generic for Intel video chipsets but I can't see much about its use with 8.04, will have to do some more research.

Capabilities: <access denied>


Need to see what driver is in use with 8.04, do sudo lspci -vv and you should get a print similar to the 10.04

Or try sudo lspci -k which should specifically show kernel modules for each device

Failing that post /etc/X11/xorg.conf

lsmod says

rtai_smi 3600 0

So I guess it is loaded, but it's not used by anything. Or does this mean, that it is not loaded?


The actual chipset is ICH7 Rev 02 although it refers to 82801GBM/GHM in places
This should not require the SMI fix, but it is loaded
If it was a SMI problem it would happen in 10.04 too, so we can eliminate that one

The 0 refers to the other modules that have dependencies with it, if you do a lsmod and look at the list, where there is a number above 0 in the right column
the modules listed in modules.dep for that module are listed next to it.

Need to find which driver 8.04 is using - looks like that may narrow down the problem

I will look into i915 availability for 8.04

regards

Please Log in or Create an account to join the conversation.

More
02 Nov 2013 00:19 #40496 by ArcEye
Hi

I have just booted a partition which has 8.04 on it.

For some reason lspci does not show which kernel module is in use for the VGA controller, hence your problems

However I have found from lsmod
i915 44544 2
drm 110304 3 i915

so it is using the i915 driver

If you can confirm this on your machine, I can only suggest that you run apt-get update and then update all the files with apt-get upgrade and see if a later driver version solves anything.

regards
The following user(s) said Thank You: maexbecks

Please Log in or Create an account to join the conversation.

More
04 Nov 2013 16:43 #40534 by maexbecks
My output is:
i915 28416 0
drm 73620 1 i915

I did the upgrade before, does not help either. Could it be a process or module that is interrupting? I can remember that I once updated LinuxCNC to 2.5 on 8.04 and then the jitter was about 15K ns. Unfortunately after rebooting my machine the jitter got high again.

Thank you very much for your help so far!

Please Log in or Create an account to join the conversation.

More
05 Nov 2013 00:05 #40540 by ArcEye
Hi

The next 2 things I would try are these

Do them one at a time, if 1) does not assist revert to hardware openGL, then try 2) and then if necessary try 1) again once 2) is done

1) Install the software openGL libraries in 8.04 - this should sidestep any openGL issue with the i915 driver
(You could try this with the 10.04 install too, to see if it cures the lock ups)
wiki.linuxcnc.org/cgi-bin/wiki.pl?Troubl...oftware_based_OpenGL

2) Install the vesa driver with 8.04
As it is 8.04, /etc/X11/xorg.conf actually exists, which makes it simpler, albeit it won't have much in it
wiki.linuxcnc.org/cgi-bin/wiki.pl?Troubl...g#Using_Vesa_Drivers

regards

Please Log in or Create an account to join the conversation.

More
05 Nov 2013 16:56 #40546 by maexbecks
Does not help either :(
But I will try the software openGL libraries on 10.04.

regards, Maxim

Please Log in or Create an account to join the conversation.

Time to create page: 0.084 seconds
Powered by Kunena Forum