LinuxCNC + Robot Operating System + OpenCV

13 Jun 2012 15:53 #20874 by tom.larkworthy
tom.larkworthy created the topic: LinuxCNC + Robot Operating System + OpenCV
Following on from I am planning on integrating robot operating system (ROS ) with linuxCNC.

My personal motivation is that I have a crap CNC conversion that drifts over time (no linear bearings) due to sticking. On other projects I have developed some very accurate optical localization routines, so I can get rid of open loop stepper control and wish to swap in optical positioning (using software already compatible with ROS). See .

Anyway, I will setup a git repository and stuff over the weekend. If anyone is interested in joining this effort, just having some testers would be valuable, I will briefly describe some of the gotchas with ROS you need to bear in mind before installing anything.

You *need* ubuntu 10.04
You will probably find the latest version (fuerte) unusable. Install electric or diamondback.
You also need 32bit ubuntu 10.04 because I am having problems getting linuxCNC working with my 64bit system.

ROS has an awesome package installation system, so ROS will satisfy the python dependencies etc.

ROS is both a language agnostic communication system AND a build system (and logging tools and introspection etc.). Once you have a ROS system setup it is easy to link to openCV, and you should set those systems up through the ros documented methods (not "sudo apt-get" although sometimes that is the suggested method).

My optical system also uses the
vision visp
package for optical tracking (and camera calibration)
and the
module for streaming camera images
13 Jun 2012 16:05 #20876 by cncbasher
cncbasher replied the topic: Re:LinuxCNC + Robot Operating System + OpenCV
thanks for the heads up with your projects and ROS etc ,
i'd be happy to work on this too

13 Jun 2012 16:25 #20877 by andypugh
andypugh replied the topic: Re:LinuxCNC + Robot Operating System + OpenCV
I don't know if anyone else has noticed, but most of the folk discussing this and OpenCV are in various parts of the UK.
14 Jun 2012 11:38 #20911 by cncbasher
cncbasher replied the topic: Re:LinuxCNC + Robot Operating System + OpenCV
ok as a heads up , iv'e looked further and it would seem at least with my build here , the ROS librarys are incompatable with Linuxcnc , i'm getting dependancy issues , so probably the realtime kernel is being a pain ,
14 Jun 2012 19:47 #20932 by tom.larkworthy
tom.larkworthy replied the topic: Re:LinuxCNC + Robot Operating System + OpenCV
I am having problems in a different modality. I installed vanilla 32 bit ubuntu and installed the emc RTAI + EMC (that works), but it took down my ethernet :*(

The instructions that worked were:
and not these instructions:

For some reason my ethernet is not obtaining an ip address or using routing tables anymore so it might be fixable.


* So we are all UK. Cue colloquial slang for the rest of the thread.
14 Jun 2012 23:00 - 14 Jun 2012 23:02 #20941 by tom.larkworthy
tom.larkworthy replied the topic: Re:LinuxCNC + Robot Operating System + OpenCV
Bad news: can't get my ethernet working, but that's an orthogonal problem to ROS linuxCNC intergration
Good news: I have ROS running alongside linuxCNC with the RTAI kernal. I have tested as far as streaming usb camera info at the same time.

So perhaps @cncbasher ROS will not go alongside the liveCD version only??

My installation was as follows

fresh 32bit version of ubuntu 10.04

ran from
(NOT instructions from the wiki they did not seem to work for me)

Then I could restart and boot and seemingly run linuxCNC, but no internet so I booted back into normal pae kernal to install ROS electric :

To test usb camera I already have my code written from the blog, I have shared that in github

mkdir ~/ros
cd ~/ros
git clone git://

also get other peoples code out of their repositories for

adjust ROS_PACKAGE_PATH to /opt/ros/ to point to where external ROS packages are stored:

export ROS_PACKAGE_PATH=/opt/ros/electric/stacks:/home/YOURHOME/ros

then make sure bash is aware of change
source ~/.bashrc

roscd linuxCNC_ROS
should now go to the git repository using the ros tool to cd into installed ros packages (it searches the ROS_PACKAGE_PATH)

rosmake (the build tool) in that directory will fail because the dependencies are not installed

rosmake --rosdep-install

will find also install all the required extra system dependencies

now you should be able to test the usb camera (if you have on plugged in).

roslaunch linuxCNC_ROS camera_view.launch

(launch files are xml descriptions of what modules to run, and how to connect them). Its very human readable, you can see them all in linuxCNC_ROS/launch

there are a few others but it seems they are not properly working on my fresh install but they shall be an easy fix.

OK so I will spend a few days trying to get my damn Ethernet working as I can't program very quickly without internet.
Last Edit: 14 Jun 2012 23:02 by tom.larkworthy. Reason: formatting
16 Jun 2012 00:54 #20975 by tom.larkworthy
tom.larkworthy replied the topic: Re:LinuxCNC + Robot Operating System + OpenCV
OK so I am looking at the HAL system to work out where we should integrate. "halcmd list all" is actually very like "rostopic list" (and other similar functions in ros), the two systems have quite similar architectures.

As far as I can tell, its the stepper motor that provides position feedback to the motion controller on pin "motor-pos-fb" (the motion controller also commands the stepper motor over motor-pos-cmd line). I tried manually disconnecting the signal and setting the values of motor positions using halcmd and the GUI did indeed jump the end effector, but then the system errored saying the joint was faulty. I assume thats a safety mechanism, I would like to know where its error tolerance is set on jerking the position feedback signal as we may need to lower that.

Optical feedback only comes at 30fps, it is a 6dof estimate of position + rotation. The motion controller needs signals alot faster that 30fps.

I have had a few ideas for integration. The issues to deal with are
1. latency on camera signals
2. slowness of camera signals
3. The existing motor fb signal will never get reset by the camera system, so could drift to ridiculous values

My best idea so far is when a camera frame comes in, we take the difference between its estimate of position and the motor pos fb value to create an error term. This error term can just be added to motor-pos-fb and fed into the motion controller. However, that probably won't work because of sharp changes in the error term (exacerbated by camera latency), so **we need a better way to smoothly alter the error term** (ideally without too much logic)*

Later we might want to linearly predict the expected camera signal based using latency, but not initially.


* my very first idea was a weighted sum between camera estimate and motor-pos-fb, but then I realised when the motor drifted alot the whole thing would be out of kilter all the time. The motor-pos-fb needs to be considered just a differential quantity rather than an absolute one (whereas camera-fb is our golden absolute feedback).
16 Jun 2012 01:01 #20976 by andypugh
andypugh replied the topic: Re:LinuxCNC + Robot Operating System + OpenCV
I am confident I could answer your questions if I knew what they were.

In the HAL file try "show all" after the loadrt section.

You should consider short-circuiting motor-pos-cmd direct to motor-pos-fb. f-error limits are read into Motion from the INI. You might not need Motion, depending on the application.
19 Jun 2012 12:28 #21095 by tom.larkworthy
tom.larkworthy replied the topic: Re:LinuxCNC + Robot Operating System + OpenCV
Yeah I am sure that last post looked garbled. I was just thinking out load really (at 2a.m.). So yeah, I think I know what I need to do next.
26 Jul 2012 02:08 #22471 by johnjsb
johnjsb replied the topic: Re:LinuxCNC + Robot Operating System + OpenCV
i am so glad someone else is wanting to use ros and linuxcnc together. the idea is to use a omni robot use gcode to make a design out in the middle of a room or something. use the visual servoing to correct position info. there are several ways to do this but resently useing 2 kinects at x and y fixed to the room. since we all have tonnes of metal shaveing laying about a industrial grade roomba that cleans and deposits swarf would be great. so close
Time to create page: 0.163 seconds
Powered by Kunena Forum