OpenCV vision with EMC/linuxcnc
I'm really sticking my neck out here, but I'd like to start the ball rolling with the pick and place guys... I'm a vision systems designer and I'm familiar with openCV, I've got an engraving machine and use EMC. Combined with the fact I design and build a lot of high spec electronics, a pick and place machine seems a logical progression.
I've read as much as I can find on integrating vision into EMC, I'm keen to make a start but I'm not familiar enough with EMC's structure to get things going.
Is there anyone out there who is willing to collaborate on this work? Ideally any willing volunteers would have an in depth knowledge of the core of EMC. I'm happy to contribute the eventual works back to the EMC/linuxcnc community.
I hope we can make a really solid platform for intelligent vision systems in EMC, the pairing of OpenCV and EMC seems to good to overlook.
My email is This email address is being protected from spambots. You need JavaScript enabled to view it. - I'm keen to hear from anyone who's interested in helping out...
Best regards,
Ed
Please Log in or Create an account to join the conversation.
Does the openCV just send a simple signal back to linuxcnc or complex movement commands?
Please Log in or Create an account to join the conversation.
Does the openCV just send a simple signal back to linuxcnc or complex movement commands?
I imagine it would provide measured positions in a HAL component, for example fiducial line co-ordinates.
But I agree, the first part of the puzzle is to work out what the camera is for.
Please Log in or Create an account to join the conversation.
cmorley wrote:
Does the openCV just send a simple signal back to linuxcnc or complex movement commands?
I imagine it would provide measured positions in a HAL component, for example fiducial line co-ordinates.
But I agree, the first part of the puzzle is to work out what the camera is for.
This is exactly why I thought I should publicly announce my intention of working on this - the ideas will hopefully be free flowing and we can quickly pick a sensible route.
I can write imaging routines that will find chosen objects in machine coordinates, i.e. fiducials, SMT components etc. I'd also propose functions for measuring rotation of parts for pick'n place.
I'm flexible, can implement this in any way that suits EMC and our needs - I'd just like some clear direction before I get too busy with it.
Building numerous small functions into a HAL component seems the best way to go - looking at the HAL layer, it's seems ample for my needs.
Thanks for looking...
Ed
Please Log in or Create an account to join the conversation.
I can offer first hand insight to how MY commercial PnP works. It seems the vision section provides offsets to the control for 2 things -
1) when a new board is loaded, the control sends the fudical camera over to "measure the board using the board fudicals. This in turn gets recorded as a global offset for the session. It easures both X and Y position, also theta or board rotation.
2) There is a second (and in my case a third for larger parts) camera that takes a picture of each component before it gets placed. This part of the system compares the part to a package definition for the part being placed and in turn seems to provide offset info simular to the 1st step; X,Y and Theta. As a secondary thing, it also confirms there is a part and the part is good. ie: if a pin is bent outside the package def it will trash the part and get a new one.
A good PnP should have 2 cameras to work as described above.
Now in a milling or lathe operation, Assuming we can achieve the same accuracy as my PnP, I could see a camera checking parts before maching takes place much like step 1 above. This would be a HUGE HUGE thing in the machiine industry. I can't tell you the number of parts I have had to scrap simply because an operator loaded it wrong and hit go.
Next could be a visual inspection function, like a poor mans CMM. Here the vision system would just provide coordinates of features that can be either compared to a referance or spit out an inspection sheet.
In a lathe, diameters and features could be checked and maybe re-cut if not up to snuff. Again here the camera would provide dimensions.
And last thing I can think of for now, a simple tool height gage/tool checker. Here the camera would be mounded in a fixed location and the tool presented to it. If its a new tool, the tool offset would be calculated. If it isan in use tool, it could be looked at to see if broken or worn.
So based on this long post - my opinion to the question of "what will the camera be for", is a vision system able to provide measurements/dimsions in what ever form LinuxCNC needs them. At a minimum this will be X,Y. But woud be GREAT if theta can also be done.
I am more than willing to help in any way I can to help with this goal. I think this could put LinuxCNC so far ahead of every thing else (not that it isn't already).
Please Log in or Create an account to join the conversation.
Now in a milling or lathe operation, Assuming we can achieve the same accuracy as my PnP, I could see a camera checking parts before maching takes place much like step 1 above. This would be a HUGE HUGE thing in the machiine industry. I can't tell you the number of parts I have had to scrap simply because an operator loaded it wrong and hit go.
A camera inside of a VMC or a Lathe would last about 30 seconds till coolant got on the lens and it could not see again. I've installed cameras on production lines and they are a constant maintenance headache. One camera we installed in a "cleaner" area of the production line lasted 2 shifts before they turned it off... turns out the operators would put a mark on the part up stream for some reason and when it got to this station they squirted it with alcohol to clean the mark off. The splatter covered the lens in no time.
John
Please Log in or Create an account to join the conversation.
![:) :)](/media/kunena/emoticons/smile.png)
You mean since I can't even see through the doors, The inside mounted camera would be blind also? Hmmm thats a tough nut to crack. Now you got me thinking about scara mounted camera to move in when needed much like my touch probe does on my lathe..
Please Log in or Create an account to join the conversation.
A camera inside of a VMC or a Lathe would last about 30 seconds till coolant got on the lens and it could not see again
It might be possible to cover the lens except during tool changes.
I am actually curious about how good a camera + vision system would be as an automated tool-length measuring system. It seems stupid on the face of it, but webcams are so very cheap.
Please Log in or Create an account to join the conversation.
In any case we try and talk the customer into anything but a camera as a solution when possible due to the maintenance of keeping the camera up and running.
John
Please Log in or Create an account to join the conversation.
The cameras we use have all the software built in and use a network cable and your browser to set it up.
And a Playstation controller?
I worked for a company that was an integrator for the Keyence cameras, and did the training course, but we never installed one as far as I recall.
Please Log in or Create an account to join the conversation.