How would you design a real time application for cnc control in c / c++ ???

Hi,

This question is how would you design a real time application with the possibility to run real hardware trough the golden ring?

With real time i mean the core of the application, like motion control is capable of running at a cyclus time less then 1 millisecond including all the needed calculations like interpolation etc.

I did a linuxcnc latency test for my home pc :

The conclusion that i took from this test is that this pc is not usable for linuxcnc.
The bad latency can be related my current kernel 4.19.0-0.bpo.9-amd64, i needed this kernel to support the displayport.

I was curious how other's are designing real time applications. But there is not a lot of info around, except Linuxcnc wich is quite
complex to figur out.

My idea is to:
- design a software module that is in fact a core that uses diffenent executable programs trough :

1. centralized shared memory, to talk with the gui, motion control, i/o, etc.
Not like the lcnc hal layer, but pure software communication.

Why this? I could not find a faster way of communication then using shared memory. So i think it will be nice to base the design
on this type of communication. In my opinion it looks like using extern variables, but then trough different programs.

I did a short test to run a real time app on this bad latency pc. Its programmed to do what it can's within 1 millisecond.

The output was not as bad, the code between the spoiler is executed 16 times within 1 millisecond.
Now this is a very ugly and short program example, but it can write shared memory, do a few calculations, update a few functions, etc.
within a nice time. It's just a test example.

So i think i would design it maybe around a stand alone program that acts as core and respects a real time clock speed.
And talks to other executable programs like the gui, a executable program for motion control etc trouth shared memory.
How would you do it?

Your post is very interesting I have experience using Labview Real time with NI-Realtime target, and also simulink Real time target, to run realtime models from simulink and data aquisition, on that project the comunication between targets was performed bi GE reflective memory comunicating models and NI-PXI by opical fiber ring about 10 PXI and 14 simulink RTT, this same aproach is done RTAI-LAB however I never tested,check tis document

www.rtai.org/userfiles/downloads/RTAILAB/RTAI-Lab-tutorial.pdf

I have thinked for a while in the posibility of reproduce something similar using HAL components and raspberry pi as real time tergets using gigabit network instead of optical fiber. for a Hardware in the loop tests.
not sure about your project but if you post more info may be i can help (or maybe not) but i interested in know more about

linuxcnc and HAL use a shared memory model for realtime communication.
What are you proposing to do different?

@Grijalvap,

I have not quite a lot of info at the moment. Just asking how other people are thinking about desinging real time software.
Linuxcnc is designed a while ago. I was wondering what the designers would do different if designing it from scratch this day's.

not sure about your project but if you post more info may be i can help
It's all very new for me. So i will try to make some example soon where you can look into. I hope to provide a modulair example.

@Chris,
What are you proposing to do different? That item is not different i think.

I imagine the core is a stand alone c++ program, that acts as a distributor for read and write data trough shared memory and
repects a real time clock sequence. It load's and unload's also the .ko files. (for readers : .ko = kernel modules like i/o card drivers.)

For real time app's i think you need a boss that takes responcebility for the clocktime.
If subroutines do not complete within their task time, there is a problem i think.

The whole program will be not so big in code. I think a max of 2000-4000 lines to ensure the the operating speed at older pc's.
It is much smaller then Linuxcnc. Starting with a lightweight, hopely ultrafast app.

But still everything is unknown. I hope to get some good idea's around here that can help.

If you lean towards several tasks have a look at patents.justia.com/patent/4908746 and catch the idea of semaphores in the shared memory...
Mike

I think the Linuxcnc model is quite clever. Complex but clever.

Somewhere at its core, there is an interrupt service routine that fires every millisecond, We call it the servo thread. Study up on interrupt service routines (ISR's), particularly timer interrupt routines. When you write one of these, your routine takes over the interrupt and when its finished chains to what was there previously. This creates an ISR chain

Then LCNC's component architecture allow you to write your own components which are added to the ISR chain in the HAL file.
Then there is a series of totally seperate processes that communicate through shared memory.

I think if you were to try building a new system, use the Linuxcnc interpreter and work on the trajectory planner to keep the size of the project manageable.

If anything Linuxcnc suffers from old code maintained by a lot of people that predates C++ object oriented code. Some code is in C and some is now in C++. Come code is poorly structured.

Hi Mike,

During watching youtube video's about writing kernel modules. There are example's how to lock the task with the use of semaphores to prevent multiple clients try to connect to the same module at the same time. mutex is a variant of semaphore.
So far i did not need them.

Hi Rod,

Complex but clever. I agree.

Somewhere at its core, there is an interrupt service routine that fires every millisecond, We call it the servo thread. Study up on interrupt service routines (ISR's), particularly timer interrupt routines. When you write one of these, your routine takes over the interrupt and when its finished chains to what was there previously. This creates an ISR chain
So far i know interrupt's are used by kernel modules to prior a certain task. The real time program at user level does not use interrupts, or am i wrong?

I think if you were to try building a new system, use the Linuxcnc interpreter and work on the trajectory planner to keep the size of the project manageable.
This is interesting.

You didn't know yet. But i was already started coding a trajectory planner for gantry kinematics. The hardest part for me is was to figur out acceleration and de-acceleration during traject. But i did my first real time test tonight for one axis, and it already looks like a fanuc controller.

But no wonder if you have a lightweight core that is capable to update 2 or more times a millisecond with current code load. How huger the core code, how slower it will be, the slowest senario will be 1 millisecond with no micro-second left in space.
So we keep the core as tiny as possible. At the moment i go out from a fixed cyclus time of 1millisecond. Within this time all real time code has to fit. Space left within the 1 millisecond threat is room for extra code. So it's easy to see when you are at the limit through
the chrono timer.

I am not really afraid of the size of a project. Once the real time core works, and a gantry trajectory planner works, it's not that hard anymore. But i agree with you.

If anything Linuxcnc suffers from old code maintained by a lot of people that predates C++ object oriented code. Some code is in C and some is now in C++. Come code is poorly structured.
All kernel code is in C, combined C and C++ at user level. I did intensive coding at linuxcnc for a few day's.
I have respect for the linuxcnc source code. Each time i learn more about it. Like using char drivers. First i thought, what is this?
Use srings, no char's please. But kernel land uses char drivers instead of slower block drivers.

At the end of the tunnel i see already a Ethercat light, and a FPGA light. Each day comming closer to real hardware.
The goal is a modulair piece of software. Labview is a example, Linuxcnc is a example. I want to do something different, but
must be simple so everyone understands the straight forward code style.

I hope that grijalvap can use the code for his data aquisition projects. Have a nice day and keep optimistic about coding.

The Dutch goverment has placed a tracer on a russian ambassy car. In Holland a few guy's are really sick !!
I hope mr. Putin tell's them not to fuck with the Russian's !

Okey here we have 2 program's running. The gui start the core. When gui is shutdown. Shared memory is cleaned up. This is
really importand. Cleaning up the shit.

1. The core (real time)
2. The gui (refresh rate 0.1 sec for motion, 0.1 sec for opengl, 0.1 sec for mainwindow)
The core does takes responcebility for motion, like acceleration, deacceleration, update kernel drivers in the future.
The core is connected with the gui with shared memory. At startup we create 100 shared memory channels at once with a size
of 1024 chars. But i have thought to decrease this where possible in the future .

github.com/synthetos/TinyG/wiki/Jerk-Con...led-Motion-Explained


you can take a look to this project it is interesting and have jerk control

Grotius wrote:

I was wondering what the designers would do different if designing it from scratch this day's.

Being a software developer by trade, these are the most glaring things I'd do different:

* Data encapsulation: Shared memory is okay for communication I guess, but the way LinuxCNC does it is absolutely not up to modern standards: LinuxCNC stores objects themselves (including internal state) in shared memory so that they are exposed read-write "to the whole world". Internal state should be stored locally/inaccessible to the outside, and then you create some kind of protocol for how to share data, be it by shared memory or by some other mechanism. The data that is to be shared should then be copied to and from the shared memory according to that protocol.

Without encapsulation you cannot test properly and code maintenance becomes very difficult.

* I would make the program user-mode only: no kernel modules. Modern Linux (and probably other OS:es) support this well enough today that it isn't worth the huge extra complication to hook into the kernel.

* I would chose a modern programming language that supports object oriented programming and enforces encapsulation and memory safety among other things. Rust comes to mind; unlike many other modern languages it's also time-deterministic (no garbage collector that kicks in now and again), which should fit well for real-time use. I'm sure there are other options as well. (Most of my programming I've done in C; it's the language I know best, but honestly it shows its age and it cannot support modern programming practices. C++ is a behemoth a la Frankenstein's monster IMO, and there's no memory safety.)

Also note that "small" is not equal to "fast". The two are unrelated.

There is something seriously wrong with the test program if it only manages to put in 16 iterations in one ms.

CPUs nowadays run at 2GHz+, so that amounts to 125000 cycles per iteration doing essentially nothing. I suspect std::cout << is slowing this down.

If I were to design something like linuxcnc from scratch, I would first try to get the IPC layer sorted. If at all possible, use/adapt something existing like zeromq, else you are still debugging concurrency issues 5 years from now in 5 different programming languages.

Avoid kernel modules at all cost. Exception may be step generator, but I would implement that as a device driver, and keep the rest of the system in userspace. You can't easily debug kernel modules, most programming errors need a reboot, even a recovery from backup could be needed in some extreme cases.

You will end up with something similar to linuxcnc.

If you have too much energy, that would be better spent in e.g. replacing NML layer in linuxcnc with something modern, port gtk2 to gtk3 stuff, python2 to python3, update classicladder, yadda yadda yadda.