Closed Loop Computer Vision Enabled CNC

Hello, I have been using opencv to perform just regular tracking but I have no idea how to implement the opencv tracking algorithms in a cnc machine. Aciera has given some awesome ideas for doing the closed-loop part, how does your implementation work?

Hello again, what is the difference between using hal and using Dmitry's pick and place machine? I have been briefly reviewing Dmitry's honestly incredible machine and it is frankly very complicated for me given my inexperience although I am hoping to obtain the necessary experience for my own project.

Might be something here worth looking at

Hello,

Thank you so much for all the help. I just wanted to confirm that applying external offsets through a custom HAL with the data obtained from a visual tracking algorithm doesn't require any additional hardware, i.e., external controllers. The only reasons I want to be 100% sure is that I currently run my cnc using mach4 and it has been made clear to me that mach4 would require an additional controller to perform what I am looking to achieve. I reached out to a motion controller company and the cost for customizing their motion controllers was very expensive. So, before I transition to LinuxCNC and overcoming the learning curve, I wanted to confirm with you if my project could all be achieved purely with software on linux? Thanks

I can certainly confirm that machine vision can be used with external offsets without any extra hardware (except the camera of course). Be aware though that python may not be fast enough for your application so you may need to use the c++ library of OpenCV (if that is what you are aiming for) and somehow stream the correction offsets to a LinuxCNC real-time component (which will need to be in C).
This may be of interest:
linuxcnc.org/docs/stable/html/man/man9/streamer.9.html

Be advised though that there may not be much help available regarding the actual implementation of your code so you would have to be fairly confident with programming or willing to do a fair bit of your own research. Of course we always appreciate project reports.

Thank you. Without your incredible help, I would still be searching for how to implement this project. My current CNC machine is a 5axismaker with Mach4 software and Pokeys57U hardware. After reaching out to the vendor, they confirmed it can be converted to run LinuxCNC and suggested I follow this post: forum.linuxcnc.org/24-hal-components/29816-pokeys-ethernet . My first step will be converting the machine to LinuxCNC as well as figuring out LinuxCNC. Then, I'll create a custom HAL to apply external offsets to the original g-code using data from the camera using OpenCV. I feel like I've done adequate research beforehand but the actual implementation is where I will see how difficult this gets. While I'm not as experienced with CNC control, I'm more comfortable with programming but we will see. Hopefully, I can show the completed project as soon as possible image missing /media/kunena/emoticons/smile.png!