Writing modules for PREEMPT_RT

Within a couple of weeks there will be the yearly meeting of CNC-enthusiasts. The last two meetings I spent an hour in front of the beamer telling about LinuxCNC. There is always a lot of interest in it and the classroom is full of people, but Linux is scary, wires are scary, and listening to someone talking for an hour doesn't remove the 'scaryness'.

This year, I would like to take it one step further, and possibly make it some sort of workshop, starting out with a fresh install and definitely ending with something moving, because moving motors and controlling lights with a computer has some magic in it and is highly rewarding.

Now, I intend to have people bring their own laptop and have them buy a little bit of hardware. That assures me that people showing up are motivated, and doing is the best way of learning.
I am considering the STM32F103 Minimum System Development Board as the motion controller, and a few cheap StepSticks or so to drive motors. I always have plenty of those little STM boards in stock to solve little problems so for me that's an obvious choice.
Total hardware cost excluding motors is $12 or so, affordable for everyone. Motors are easy to borrow, or must be bought anyway when someone wants to build a machine.

But, a big 'but'...
When someone wants to use a laptop and possibly a virtual machine running LinuxCNC, there is only one suitable communication bus: USB.

Now, loading some HID-class code, stepgens and a few digital I/O's into the STM32 wouldn't be that much work. I have once tried the custom HID framework provided by STM, and my laptop recognised the device. Cool.
Writing a userspace module that talks to the device would not be a huge problem either; plenty of sample code available. Latency is not such a huge issue since communication is mainly one-way (so a little timestamping helps a lot)., so I guess it would work. Maybe with a hickup here and there, which I think is totally acceptable for a 'get people interested' demo.

But this also sparks my interest. How unbounded is the USB latency, really? Interrupt transfers to/from a fullspeed device happen every millisecond. Without RT extensions there is no guarantee at all, but that is where PREEMPT_RT comes in. It seems Mesa does a fair job dumping UDP packets to their hardware in a timely manner, and when scanning the hm2_eth code diagonally they seem to use the regular IP stack. As a dumb hardware engineer who is not hindered too much by knowledge of software it makes me think 'If it can be done with Ethernet, then it should also be possible with USB. And if we can get a reliable 2-4msec turnaround time, it would be useful for a LOT of things.

So, I'd like to try PREEMPT_RT to satisfy my own curiosity.
Now, how does one write realtime modules for PREEMPT_RT that use userspace libraries? Can I still use halcompile? I love halcompile, makes code very clean and to the point. If I use hm2_eth as a template and keep it pure C, then what is the easiest way to turn that into a module?

I think the issue with USB is that unlike Ethernet, AFAIK the USB driver has no RT patches, so will still have relatively bad latency even with a RT kernel. Another issue with USB is that if you have re-enumeration you have have _long_ periods of USB activity pauses.

_But_ for a demo or a simple machine, I think you can work around most of of these issues by putting enough smarts in the remote USB device. For example, our Ethernet remotes use a DPLL to lock onto the access rate to remove sampling jitter from encoders an stepgen position counts (on Ethernet this jitter may amount to 50 to 500 usec peak depending on the host), I think a similar method could be done with USB. Also if the stepgen implemented acceleration, then the (necessarily lower) update rate of USB (maybe only 200 Hz or so) could still generate clean pulse streams without excessive velocity step sizes during acceleration.

(I am not suggesting any multi cycle buffering as that's contrary to the design philosophy of LinuxCNC and would likely require structural changes in LinuxCNC itself if a reasonable feature set is to be retained)

I do not consider USB as a viable communication interface for 'real' machines and I have no intention to design a USB-connected motion controller for LinuxCNC that can actually be used for real machines.
Besides possible software-related issues the major drawback of USB is that it just isn't stable in an electrically dirty environment. Adding a Chinese VFD to a USB-connected gantry mill is often quite enough to cause problems, especially when grounding topology is less than optimal (which it usually is). Adding bus isolation helps, but this raises cost and complexity into the realm of Ethernet, so that's useless too. If people decide to use LinuxCNC in a real situation I recommend Mesa hardware anyway.

For the hands-on workshop I would really prefer to have people use LinuxCNC in a virtual machine on their own laptops. I cannot control the computing hardware, but at least I can control the contents of the VM which makes the environment a little more predictable. I have yet to see that working sufficiently well though.
Plan B would be a bootable USB flash drive. But take 10 laptops and at least 2 of them will have problems with that.

This also implies that using a RT kernel makes no sense at all and a userspace module will probably handle the job.

I indeed intend to communicate position,velocity and acceleration. Not too interested in a DPLL for this however; if it proves to be needed a small multicycle buffer is even more effective. It increases roundtrip delay, but in this case communication is mainly one-way and my latency needs are 'not too obvious for humans'. No real machine requiring micrometer positioning accuracies will be controlled. Please don't shoot me if I decide to implement a several msec ringbuffer to cope with wildly varying latencies.

But then there is still that nagging voice in my ear saying 'Is it so bad? Is it? How do you know? Do you thrust everything they write on the internet?', hence my interest in PREEMPT_RT. Would make the coding somewhat more interesting too since I have the opportunity to venture into unknown territory.

If its a one directional control with no "real time" monitoring then sure a buffered interface will work
I have my own USB experience (we have USB interfaced FPGA motion cards) and in general USB is a
nightmare compared to Ethernet from a timing, electrical, and dreadful driver basis

I have only ever used USB for it's intended purpose: a setting here, some logging info there, nothing serious

At this moment my only intentions are to get a motor to move, with hardware that will work on laptops and which is cheap enough to buy it for fun and not cry when the magic smoke disappears. I helped a lot of people getting their CNC electronics up and running, and everyone is smiling when that motor with vane of tape attached starts to move, or when a LED lights up as a reaction to a button pressed on the screen. It is that smile I'm after. I suppose I can do that from userspace too.

I definitely want to keep the default LinuxCNC functionality intact. HAL should function normally, kinematics should function normally, HAL<->GUI interaction should work, etc.

All I need is a motion controller that follows the 'dumb hardware, smart PC' method, works with laptops and with a cost low enough to make it a toy for the day. Real machines should use real hardware.

andypugh wrote:

Have you tried it? Apparently you can make any system calls from a comp that you want, but there is no guarantees that they will be thread-safe.

Not yet. I'll try the userspace approach first. If that works I can always 'upgrade' to a PREEMPT_RT realtime component and see if that improves consistency.

I didn't spend a lot of time of coding yet; have to prepare for a motorcycle trip first.
I created a custom HID device on that STM32, usbview under Windows sees it properly, lsusb under Linux does too, but as soon as I try to access it with hidapi I get a segfault. It doesn't do that with another HID device. Oh joy.

Maybe it would be easier to drive a single stepper through the speaker port?

I have used that speaker port as an interface before, but there are plenty of issues with it. They certainly cannot do DC or very low frequencies, so directly driving stepper motor windings with an amplified audio signal is not a good idea. One could also use the speaker port as a data port by modulating bits on a carrier. But for any data rate above a few kbit/s one needs far more
complex modulation methods than simple 2FSK/AM/2PSK/etc, stable clocks, and the latency of regular audio is in the 100's of msec range.