Preparing native Raspberry Pi OS for LinuxCNC
- ago_tm
- Topic Author
- Offline
- Junior Member
Less
More
- Posts: 29
- Thank you received: 3
23 Feb 2024 18:19 - 23 Feb 2024 18:38 #294098
by ago_tm
Preparing native Raspberry Pi OS for LinuxCNC was created by ago_tm
Hello,
I successfully finished with the experiment whether could I reach the same performance as LinuxCNC 'native Debian 12 Bookworm' image but using a regular Raspberry Pi OS (64bit) on Raspberry Pi 4b.
uname -a
# Linux pi-cnc 6.1.77-rt24-v8-ago+ #1 SMP PREEMPT_RT Thu Feb 22 01:29:42 MSK 2024 aarch64 GNU/Linux
latency-histogram --nobase --sbins 1000
https://github.com/golyakoff/linux-info/blob/main/img/latency-histogram.png?raw=true
I received maximum jitter value ~80 μs (80,000 ns) with 8 `glxgears` (and it is with out-of-the-box vnc server working over wifi).
I prepared a step-by-step instruction for building the RT-patched kernel (based on Raspberry Pi OS 64bit) on my GitHub page github.com/golyakoff/linux-info/blob/mai...aspberryOS/README.md.
I am not sure if it is better to post here all the text (please let me know)
Hope this will help someone, thanks!
I successfully finished with the experiment whether could I reach the same performance as LinuxCNC 'native Debian 12 Bookworm' image but using a regular Raspberry Pi OS (64bit) on Raspberry Pi 4b.
uname -a
# Linux pi-cnc 6.1.77-rt24-v8-ago+ #1 SMP PREEMPT_RT Thu Feb 22 01:29:42 MSK 2024 aarch64 GNU/Linux
latency-histogram --nobase --sbins 1000
https://github.com/golyakoff/linux-info/blob/main/img/latency-histogram.png?raw=true
I received maximum jitter value ~80 μs (80,000 ns) with 8 `glxgears` (and it is with out-of-the-box vnc server working over wifi).
I prepared a step-by-step instruction for building the RT-patched kernel (based on Raspberry Pi OS 64bit) on my GitHub page github.com/golyakoff/linux-info/blob/mai...aspberryOS/README.md.
I am not sure if it is better to post here all the text (please let me know)
Hope this will help someone, thanks!
Last edit: 23 Feb 2024 18:38 by ago_tm. Reason: broken markup
The following user(s) said Thank You: elovalvo
Please Log in or Create an account to join the conversation.
- rodw
- Offline
- Platinum Member
Less
More
- Posts: 10700
- Thank you received: 3530
23 Feb 2024 20:43 #294114
by rodw
Replied by rodw on topic Preparing native Raspberry Pi OS for LinuxCNC
Great to see somebody else experimenting. There are a couple of other methods you can try if you like the experimentation
1. Download the Debian Image for the pi raspi.debian.net/ install a desktop using tasksel, install the Debian RT kernel from their repo, install linuxcnc using the scripted install mentioned in the 2.9 Getting linuxcnc docs
2. clone my cross compiled builder which is used to make the images available on the downlods page github.com/rodw-au/rpi-img-builder-lcnc/tree/linuxcnc-2.9.2 (get the 2.9.2 branch)
So option 2 runs on a normal desktop PC which substantially reduces compile times. It downloads the official Raspberry kernel source, applies the RT patch and builds a kernel, Then it Installs Debian with that kernel. We install the xfce desktop (from userdata.txt settings) and linuxcnc in userscripts github.com/rodw-au/rpi-img-builder-lcnc/....2/files/userscripts This installer used to build linuxcnc from source but during the 2.9 release process, a deb file was created on the linuxcnc repo so we use it now.
Have fun
1. Download the Debian Image for the pi raspi.debian.net/ install a desktop using tasksel, install the Debian RT kernel from their repo, install linuxcnc using the scripted install mentioned in the 2.9 Getting linuxcnc docs
2. clone my cross compiled builder which is used to make the images available on the downlods page github.com/rodw-au/rpi-img-builder-lcnc/tree/linuxcnc-2.9.2 (get the 2.9.2 branch)
So option 2 runs on a normal desktop PC which substantially reduces compile times. It downloads the official Raspberry kernel source, applies the RT patch and builds a kernel, Then it Installs Debian with that kernel. We install the xfce desktop (from userdata.txt settings) and linuxcnc in userscripts github.com/rodw-au/rpi-img-builder-lcnc/....2/files/userscripts This installer used to build linuxcnc from source but during the 2.9 release process, a deb file was created on the linuxcnc repo so we use it now.
Have fun
The following user(s) said Thank You: ago_tm
Please Log in or Create an account to join the conversation.
- cornholio
- Offline
- Platinum Member
Less
More
- Posts: 976
- Thank you received: 341
23 Feb 2024 21:20 #294120
by cornholio
Replied by cornholio on topic Preparing native Raspberry Pi OS for LinuxCNC
I did this about a week ago using a precompiled kernel.
One thing I did as well was change form wayland to X.
One thing I did as well was change form wayland to X.
Please Log in or Create an account to join the conversation.
- ago_tm
- Topic Author
- Offline
- Junior Member
Less
More
- Posts: 29
- Thank you received: 3
24 Feb 2024 21:37 #294262
by ago_tm
Replied by ago_tm on topic Preparing native Raspberry Pi OS for LinuxCNC
Hello! I've started playing around with point 2 from your reply - it really faster to build a kernel on 16 adult cores instead 4 children - thanks a lot, great job!
Can I ask you the questions about some parameters?
1. What are these two?
### WHOAMI AND HOST
KBUSER="linuxcnc"
KBHOST="raspberrypi"
2. Commit - it is from linuxcnc repo, right?
COMMIT="342c7ee49e862edc30c893f141f55b9211b7a43b
3. rt-patch is automatically taken fresh or it is assumed that I had to update it manually?
Thnks again!
Can I ask you the questions about some parameters?
1. What are these two?
### WHOAMI AND HOST
KBUSER="linuxcnc"
KBHOST="raspberrypi"
2. Commit - it is from linuxcnc repo, right?
COMMIT="342c7ee49e862edc30c893f141f55b9211b7a43b
3. rt-patch is automatically taken fresh or it is assumed that I had to update it manually?
Thnks again!
Please Log in or Create an account to join the conversation.
- cornholio
- Offline
- Platinum Member
Less
More
- Posts: 976
- Thank you received: 341
24 Feb 2024 23:11 #294278
by cornholio
Replied by cornholio on topic Preparing native Raspberry Pi OS for LinuxCNC
1 it’s the host and user name for the final image.
2 the commit is from the RPi kernel source
3 the script applies the patch
Rods script creates a purely Debian based image, completely different to the Official Raspberry Pi image.
The config.txt file is in a different location, some of the RPi utilities aren’t there. There’s no GUI network manager, rod claims this adds latency, but when I produced images with it included no one complained about any latency issues, apart from one user comparing their image that isolated 2 cores and the one I provided having no cores isolated. The gvfs modules I also included, again no complaints about latency issues.
My thought on the issue, the official RPi image is a double edged sword, everything is there to get up and going, but lacks some of the things included in the official Raspberry Pi Foundation images, as well as the config.txt and boot.txt being in different locations. So on one hand being an easy way to get up and going but a little confusing if the docs aren’t read.
To get an understanding of how the build scripts work you need to read through the sources, it’s not all that complicated and it’s quite modular. Getting the locales to work was a bit hit and miss to begin with.
Rod has put some really hard work into the image builder, during the initial stages we were bouncing ideas off each other, I had a few different ideas of the build process and what should be included and what not, but it was his baby from the beginning I really didn’t push the point.
It’s good and works.
2 the commit is from the RPi kernel source
3 the script applies the patch
Rods script creates a purely Debian based image, completely different to the Official Raspberry Pi image.
The config.txt file is in a different location, some of the RPi utilities aren’t there. There’s no GUI network manager, rod claims this adds latency, but when I produced images with it included no one complained about any latency issues, apart from one user comparing their image that isolated 2 cores and the one I provided having no cores isolated. The gvfs modules I also included, again no complaints about latency issues.
My thought on the issue, the official RPi image is a double edged sword, everything is there to get up and going, but lacks some of the things included in the official Raspberry Pi Foundation images, as well as the config.txt and boot.txt being in different locations. So on one hand being an easy way to get up and going but a little confusing if the docs aren’t read.
To get an understanding of how the build scripts work you need to read through the sources, it’s not all that complicated and it’s quite modular. Getting the locales to work was a bit hit and miss to begin with.
Rod has put some really hard work into the image builder, during the initial stages we were bouncing ideas off each other, I had a few different ideas of the build process and what should be included and what not, but it was his baby from the beginning I really didn’t push the point.
It’s good and works.
Please Log in or Create an account to join the conversation.
- rodw
- Offline
- Platinum Member
Less
More
- Posts: 10700
- Thank you received: 3530
24 Feb 2024 23:50 #294288
by rodw
Replied by rodw on topic Preparing native Raspberry Pi OS for LinuxCNC
Yes, I used 20 cores....
You need to read and understand userdata.txt.
There are basically 3 steps.
1. Get the Pi kernel source and build the lernel
2. Build the root file system
3. Build the image.
The relevant preempt_rt patch 6.1.69-rt21 from kernel.org is in userpatches
The commit is from the Raspberry 6.1.y kernel source github.com/raspberrypi/linux
This is the very next commit after the 6.1.69 kernel was merged into the raspberry kernel repo
This is important because without locking in the commit to get, the kernel can diverge a lot and it will become unstable with that patch
You could look for later RT patches and find a later kernel release that matches it.
After the Pi5 was released it took a long time before such a merge happened or the happened out of step. The current version is common for both the Pi5 and Pi4
Also from userdata.txt
### CUSTOM KERNEL - Comment out the one not required
CUSTOM_DEFCONFIG="0"
#MYCONFIG="linuxcnc-pi4_defconfig"
#MYCONFIG="linuxcnc-pi5_defconfig"
Once you configure the kernel with menuconfig, you can save the generated defconfig and tell it to use that. This means that on future builds, it skips menuconfig.
Also in custom.txt, try changing the image size to 12288MB. (12gb) This is how much memory is used when building the image. This might let you use the pi imager to burn the image. I found some bugs in their IMG shrink routine so made it big. I can't remember if the upstream changes have been merged in to fix this.
I installed network manager but Andy observed latency got worse so I took it out. You could add it to USER_PKGS
github.com/rodw-au/rpi-img-builder-lcnc/...ata.txt#L24C1-L24C11
I deliberately kept the image as standard as possible to stay as close to what the x86 installer offers
You need to read and understand userdata.txt.
There are basically 3 steps.
1. Get the Pi kernel source and build the lernel
2. Build the root file system
3. Build the image.
The relevant preempt_rt patch 6.1.69-rt21 from kernel.org is in userpatches
The commit is from the Raspberry 6.1.y kernel source github.com/raspberrypi/linux
This is the very next commit after the 6.1.69 kernel was merged into the raspberry kernel repo
This is important because without locking in the commit to get, the kernel can diverge a lot and it will become unstable with that patch
You could look for later RT patches and find a later kernel release that matches it.
After the Pi5 was released it took a long time before such a merge happened or the happened out of step. The current version is common for both the Pi5 and Pi4
Also from userdata.txt
### CUSTOM KERNEL - Comment out the one not required
CUSTOM_DEFCONFIG="0"
#MYCONFIG="linuxcnc-pi4_defconfig"
#MYCONFIG="linuxcnc-pi5_defconfig"
Once you configure the kernel with menuconfig, you can save the generated defconfig and tell it to use that. This means that on future builds, it skips menuconfig.
Also in custom.txt, try changing the image size to 12288MB. (12gb) This is how much memory is used when building the image. This might let you use the pi imager to burn the image. I found some bugs in their IMG shrink routine so made it big. I can't remember if the upstream changes have been merged in to fix this.
I installed network manager but Andy observed latency got worse so I took it out. You could add it to USER_PKGS
github.com/rodw-au/rpi-img-builder-lcnc/...ata.txt#L24C1-L24C11
I deliberately kept the image as standard as possible to stay as close to what the x86 installer offers
Please Log in or Create an account to join the conversation.
- cornholio
- Offline
- Platinum Member
Less
More
- Posts: 976
- Thank you received: 341
25 Feb 2024 01:19 #294309
by cornholio
Replied by cornholio on topic Preparing native Raspberry Pi OS for LinuxCNC
Small correction it’s not the amount of memory used to build the image it’s the size of the file on disk used to create the boot & root partitions and file systems in, later on during the process these are mounted, as you would a physical disk partition, as well as some of the “virtual” file systems, and qemu “boots” and your have your virtual arm64 machine, and this is the environment that everything is installed for the final file system.
Actually after the image has been finalised you can start a qemu session with it and make any fine adjustments and these will be in the image after you exit the qemu session. I haven’t attempted a full GUI session but a text mode session is fine. Of course you need to setup and mount the image and add the virtual file systems before hand, but if you look through the code you’ll see how this is done.
Basically the same as when you create a virtual machine with say for example VirtualBox. You’re asked how big you want your disk to be. Then a file is created (can be a fixed size or dynamically allocated) and when you boot partitions and file systems are created in the file.
Actually after the image has been finalised you can start a qemu session with it and make any fine adjustments and these will be in the image after you exit the qemu session. I haven’t attempted a full GUI session but a text mode session is fine. Of course you need to setup and mount the image and add the virtual file systems before hand, but if you look through the code you’ll see how this is done.
Basically the same as when you create a virtual machine with say for example VirtualBox. You’re asked how big you want your disk to be. Then a file is created (can be a fixed size or dynamically allocated) and when you boot partitions and file systems are created in the file.
Please Log in or Create an account to join the conversation.
- rodw
- Offline
- Platinum Member
Less
More
- Posts: 10700
- Thank you received: 3530
25 Feb 2024 03:19 #294322
by rodw
Replied by rodw on topic Preparing native Raspberry Pi OS for LinuxCNC
I suspect you missed the pain caused by buggy changes made by upstream that broke my build....Small correction it’s not the amount of memory used to build the image it’s the size of the file on disk used to create the boot & root partitions and file systems in, later on during the process these are mounted, as you would a physical disk partition, as well as some of the “virtual” file systems, and qemu “boots” and your have your virtual arm64 machine, and this is the environment that everything is installed for the final file system.
Actually after the image has been finalised you can start a qemu session with it and make any fine adjustments and these will be in the image after you exit the qemu session. I haven’t attempted a full GUI session but a text mode session is fine. Of course you need to setup and mount the image and add the virtual file systems before hand, but if you look through the code you’ll see how this is done.
Basically the same as when you create a virtual machine with say for example VirtualBox. You’re asked how big you want your disk to be. Then a file is created (can be a fixed size or dynamically allocated) and when you boot partitions and file systems are created in the file.
Please Log in or Create an account to join the conversation.
- cornholio
- Offline
- Platinum Member
Less
More
- Posts: 976
- Thank you received: 341
25 Feb 2024 03:30 #294324
by cornholio
Replied by cornholio on topic Preparing native Raspberry Pi OS for LinuxCNC
What changes were they ?
Even from the beginning that number referred to file size for the that the partitions and file systems .
Even from the beginning that number referred to file size for the that the partitions and file systems .
Please Log in or Create an account to join the conversation.
- rodw
- Offline
- Platinum Member
Less
More
- Posts: 10700
- Thank you received: 3530
25 Feb 2024 07:11 #294344
by rodw
Replied by rodw on topic Preparing native Raspberry Pi OS for LinuxCNC
In their changed scripts they attempted to do some maths to work out the size to use and they got it way wrong. And of course i was the crash dummy that found the bug which was painful for a few days... But they fixed it.What changes were they ?
Even from the beginning that number referred to file size for the that the partitions and file systems .
Please Log in or Create an account to join the conversation.
Time to create page: 0.088 seconds