How would you design a real time application for cnc control in c / c++ ???

arvidb wrote:

Data encapsulation: ...

That's casting pearls before swine ...
The TO already stated, that he does not care about data encapsulation.
... and when you look at linuxcnc - the python part is growing without any need and python does not know anything about data encapsulation.
Looks like you're quite alone with giving importance to data encapsulation.
Well, not really alone, as I feel the same. But I would avoid copy-tasks at all costs.
.

I would make the program user-mode only: no kernel modules. Modern Linux (and probably other OS:es) support this well enough today that it isn't worth the huge extra complication to hook into the kernel.

Guess, you never tried to do realtime tasks with linux.
Linux does not allow atomic access to cpu timer, which would be the base for timebased shedulers.
Just try to create stepper control on a rasperry without mesacard. Just using GPIO.
There are quit a lot of things you are not allowed at user space.
... and when you look at a closed loop processing - there's quite a bunch of calculations. Not that easy to make things faster ...
.

If you have too much energy, that would be better spent in e.g. replacing NML layer in linuxcnc ...

That's work without value. The NML layer is a pretty clever concept - and replacing the NML layer has no benefit on the user side. There are plenty of pretty clever concepts at linuxcnc. The downside of linuxcnc is, that many contributions have been accepted from developers that did not understand the clever concepts of linuxcnc - and for so linuxcnc has become worse and ugly with the time being.
Now the "bullshit"-parts are so huge, that its nearly impossible to cleanup.

@Grijalvap,

Thank you for your link, very interesting !

@Arvidb,

Shared memory is okay for communication I guess, but the way LinuxCNC does it is absolutely not up to modern standards: LinuxCNC stores objects themselves (including internal state) in shared memory so that they are exposed read-write "to the whole world".
They are exposed. When i include the libraries of emc/nml you can do everything.
A normal linuxcnc user is not capable of doing such interventions. Data encapsulation could be essential.
For safety channels like the ethercat (Safety Integrity Level SIL3), when is software validating SIL3?

C++ is a behemoth a la Frankenstein's monster IMO, and there's no memory safety
Yes, you are right about that. Linux Torvalds say's the same if they ask him why not writing kernel modules in c++.
I think programmers will mix c up and c++ without noticing. But i am not a expert in programming yet.

Also note that "small" is not equal to "fast". The two are unrelated.
Yes.

@Rmu,
use/adapt something existing like zeromq zeromq
Good tip, but that is something usefull when program is almost finisched.

Avoid kernel modules at all cost.
Okey.

For the ethercat, different modules are already written.
Left would be writing a kernel driver for a fpga chip, it will excist already somewhere. But at the coming developping time no kernel
modules are needed so far.

If you have too much energy, that would be better spent in e.g. replacing NML layer in linuxcnc with something modern, port gtk2 to gtk3 stuff, python2 to python3, update classicladder, yadda yadda yadda.
I'am not a python programmer and classicladder is not my thing.
An sich the NML layer works fine. I have currently two little issue with NML. One is how to free memory and close the channel at a proper way, and the other is a unknown flag to me.

There is something seriously wrong with the test program if it only manages to put in 16 iterations in one ms.
It is faster indeed.

@Reinhard,
Now the "bullshit"-parts are so huge, that its nearly impossible to cleanup.
Haha.
I think you can clean it up quite fast.

Just a idea.
It would be usefull someone provides a minimal Lcnc development version at github.

The TO already stated, that he does not care about data encapsulation.
Maybe that's a lack of my knowlegde, that has no priority at this moment.

Reinhard wrote:

arvidb wrote:

I would make the program user-mode only: no kernel modules. Modern Linux (and probably other OS:es) support this well enough today that it isn't worth the huge extra complication to hook into the kernel.

Guess, you never tried to do realtime tasks with linux.
Linux does not allow atomic access to cpu timer, which would be the base for timebased shedulers.
Just try to create stepper control on a rasperry without mesacard. Just using GPIO.

I'm not saying you can do generic step generation from user space, but as rmu says, use a specialized kernel module that is mainlined and separated from your program for that.

For step generation on the RPi I would take a look at the PWM peripheral of the bcm283x chip (for which there already is a driver in the kernel). I'm not sure the driver supports it (yet), but the chip supports "PWM:ing" of data from DMA. So maybe you could send the number of alternating ones and zeros required via a DMA channel and have it clocked out at 100 MHz (which apparently is the standard clock for the PWM peripheral). Look from page 138 in the BCM2835 peripherals datasheet .

arvidb wrote:

For step generation on the RPi I would take a look at the PWM peripheral of the bcm283x chip (for which there already is a driver in the kernel). I'm not sure the driver supports it (yet), but the chip supports "PWM:ing" of data from DMA.

The question is, is it reliable, or does it go wonkers if GPU decides to mess with cpu frequency and other stuff. With DMA, there is a chance that it loops around if something goes wrong, everbody had a PC hang while playing some music that got looped until reset, that could be "catastrophic" for a CNC machine.

For a real CNC machine, you need some external hardware in any case. So if you need reliable step generation, just include a 1€ arm cortex m0 microcontroller, that is easily enough for 6 channels and 50kHz.

Grotius wrote:

@Grijalvap,
Just a idea.
It would be usefull someone provides a minimal Lcnc development version at github.

Look at machinekit on github, that started out as a fork of linuxcnc, last I looked they tried to separate the realtime/hal from the CNC code, at least at the repository and build-level.

I'm not sure what you are trying to accomplish, but basing a rewrite of linuxcnc on linuxcnc seems kinda circular reasoning.

For a real CNC machine, you need some external hardware in any case. So if you need reliable step generation, just include a 1€ arm cortex m0 microcontroller, that is easily enough for 6 channels and 50kHz.

Base of my statement was your requirement to avoid kernel modules at all cost.
So we talk about linux only.

... and if you want smooth motions, you need timebased step generations. For one axis it is pretty easy, as most stepper drivers are limited to 200kHz. But if you want exact interpolations of multiple axis, you might reach mhz frequency.
You can't spent a cpu to each axis, so you end up with a timer irq with variable delays.
To implement that reliably you need access to the cpu timer.

On a rasperry it is easy to test, as there are bare metal libraries, that can access whole hardware without limitations.

Back to linux - you can't do timer irq stuff without using kernel internals - and timer irq is the only way to implement realtime stuff. Realtime does not mean, incredibly fast, but it means, same timing under all conditions, no matter what happens to the machine.
I don't see big difference between kernel module and device driver. Both run in kernel space.

Reinhard wrote:

For a real CNC machine, you need some external hardware in any case. So if you need reliable step generation, just include a 1€ arm cortex m0 microcontroller, that is easily enough for 6 channels and 50kHz.

Base of my statement was your requirement to avoid kernel modules at all cost.
So we talk about linux only.

That was no requirement, it was a tip to avoid the really hairy stuff (kernel modules).

... and if you want smooth motions, you need timebased step generations. For one axis it is pretty easy, as most stepper drivers are limited to 200kHz. But if you want exact interpolations of multiple axis, you might reach mhz frequency.
You can't spent a cpu to each axis, so you end up with a timer irq with variable delays.
To implement that reliably you need access to the cpu timer.

If you need "exact" interpolations of multiple axis for whatever that means, get a FPGA based step generation. Open loop stepper systems are not that accurate mechanically to begin with. Even microstepping increases positions that can be reached only about 10-fold, and then those steps are not necessarily linearly spaced. "Real" servo drives also have limited bandwith in the range of kHz, so however exact your step spacing is, the drive won't react any faster than dictated by t2hose bandwidth limitations.

On a rasperry it is easy to test, as there are bare metal libraries, that can access whole hardware without limitations.

I beg to differ. Last I looked, the real boss on the raspberry SoC is the GPU, and neither documentation of the GPU and glue logic nor appropriate libraries are available. The hardware is "funny", like the tiny UART or SPI peripheral changing baudrate when the CPU enters turbo mode, because those are derived from some CPU bus clock that gets scaled in turbo mode.

Also, your usual library targetting arduino folks isn't much help in a kernel module.

Back to linux - you can't do timer irq stuff without using kernel internals - and timer irq is the only way to implement realtime stuff. Realtime does not mean, incredibly fast, but it means, same timing under all conditions, no matter what happens to the machine.
I don't see big difference between kernel module and device driver. Both run in kernel space.

With preempt-rt, all the timing stuff happens in-kernel. Other realtime-extensions also take care of the timing stuff, they provide you an API, you create your realtime threads, give them proper priorities and deadlines, and the realtime part of the os takes care of the rest.

If you want to mess with timer-irq yourself, you are talking about implementing yet another realtime companion to the linux kernel.

Grotius wrote:

@Rmu,
use/adapt something existing like zeromq zeromq
Good tip, but that is something usefull when program is almost finisched.

I don't think so, that is probably the most essential infrastructure. You have multiple components that talk to each other like g-code parser, motion planner, the hard realtime stuff and GUIs.

You don't want to couple those components too tightly.
You can't run your GUI in the real time thread.
You don't want to run the g-code parser in the real time thread.
You also don't want to run the motion planning in the real time thread if it can be avoided.

So you need some robust communication mechanism. Inventing your own will get you something like zeromq or protocol buffers (in 5 years+ time) but no CNC control.

Of course you can get rid of the GUI and put parser and an ad-hoc motion planner into the real time loop, but then you end up with GRBL.

I beg to differ. Last I looked, the real boss on the raspberry SoC is the GPU, and neither documentation of the GPU and glue logic nor appropriate libraries are available.

Sure! Documentation suffers - no question. Even for actual SoC there's no documentation.
Anyway - I never bothered with GPU - just did some tests with CPU and compared baremetal with linux tweaks ...
Don't know, whether my raspi has a turbo mode. Its unpimped
.

If you want to mess with timer-irq yourself, you are talking about implementing yet another realtime companion to the linux kernel.

No, not at all. I maltreated a rasperry long time before I started with linuxcnc. At that time I wanted to research, how usable rasperry would be for cnc environment without other hardware ...

Hi,

I have some little news.

For a while i was trying to write a real time app from scratch. The test result's where crap.
The test's where based on how to get the maximum stepper velocity output, which is a good indicator when connected to real hardware.

In the tests the step-timing was done at user level land and was resulting in a un-stable step generation. The max stepper speed was also quite slow. I stopped this development for a few weeks.

After some time i was able to release a linux rtos ethercat distro. Then i started to test real time again.

I came to a solution that makes profit of the hal environment. This was after some trial and error the best way to go.

The step generation output is at nice speeds, and stable. The test is done by a Ethercal EL2124 (5v pulse module) connected to a stepperdriver and steppermotor.



Halscope output :

Can i read the halscope as 4 pulses in 100 us?

The files that are used for this test :
- gui (hmi)
- ethercat.hal (config file to connect pin's etc quite easy)
- ethercat_config.xml (ethercat modules setup)

The workflow that i end up with is :

1. The gui makes some hal pins for i/o with hal and then loads the ethercat.hal file.
2. The ethercat hal file loads the ethercat_config.xml file
3. The ethercat.hal file has a stepgenerator, base-thread and servo-thread.
4. When i sent the motor positon command trough the app. It run's very nice !

The app code :

Used compile flag's and include path's :
LIBS += -llinuxcnchal -lethercat -Iinclude -Isrc/emc/rs274ngc -Llib -lnml -llinuxcnc -llinuxcnchal -llinuxcncini -lposemath
INCLUDEPATH += /usr/lib/ \
/usr/lib/linuxcnc/modules/ \
/usr/include/ \
/usr/include/linuxcnc/

So far so good.
The conclusion so far as i can tell, is that this example code run's with hal as rt platform
and makes usage of the existing rt libs of linuxcnc.

*The motion planning, gcode parser, opengl machine model view, etc. is not used in this test.

I think when you expand this test code with this kinematics model :


excel template file 6 axis kinematics :
github.com/grotius-cnc/AR2/blob/master/A...nematic%20model.xlsm

It could end up in a powerfull rtos environment cq ethercat robot controller.