Preparing native Raspberry Pi OS for LinuxCNC

Bugger hey, but like I said the value isn't a "memory" value but a disk image value.

Once I get this effing diabetes under control I'll pop down in the garage and start doing some real world testing of Linuxcnc on the Raspberry Pi image and the image built via the image maker....and see if the gvfs modules and network manager are an issue, from previous experience I don't see it being such an issue to make it unusable with mesa hardware, maybe direct gpio with a base & servo thread maybe a different kettle of fish . Wont be on the mill as I don't want to delve into the control box, but on the Frankenstein Myford ML7/Super 7. Reminds me I'll have to jerry rig something up for the spindle encoder.

rodw wrote:

Yes, I used 20 cores....
You need to read and understand userdata.txt.
There are basically 3 steps.
1. Get the Pi kernel source and build the lernel
2. Build the root file system
3. Build the image.

The relevant preempt_rt patch 6.1.69-rt21 from kernel.org  is in userpatches
The commit is from the Raspberry 6.1.y kernel source github.com/raspberrypi/linux
This is the very next commit after the 6.1.69 kernel was merged into the raspberry kernel repo
This is important because without locking in the commit to get, the kernel can diverge a lot and it will become unstable with that patch
You could look for later  RT patches and find a later kernel release that matches it.
After the Pi5 was released it took a long time before such a merge happened or the happened out of step. The current version  is common for both the Pi5 and Pi4

Also from userdata.txt
### CUSTOM KERNEL - Comment out the one not required
CUSTOM_DEFCONFIG="0"
#MYCONFIG="linuxcnc-pi4_defconfig"
#MYCONFIG="linuxcnc-pi5_defconfig"
Once you configure the kernel with menuconfig, you can save the generated defconfig and tell it to use that. This means that on future builds, it skips menuconfig.

Also in custom.txt, try changing the image size to 12288MB. (12gb) This is how much memory is used when building the image. This might let you use the pi imager to burn the image. I found some bugs in their IMG shrink routine so made it big. I can't remember if the upstream changes have been merged in to fix this.

I installed network manager but Andy observed latency got worse so I took it out. You could add it to USER_PKGS 
github.com/rodw-au/rpi-img-builder-lcnc/...ata.txt#L24C1-L24C11

I deliberately kept the image as standard as possible to stay as close to what the x86 installer offers
 

Hello, today I have a pleasure to test your build system. First of all - it is really great work, thanks again!
I started with the default settings (mostly) and successfully built the image, but I cannot write it to the SD Card as RPI Imager says something like "Input file is not a valid image file. File size 3,795,000,000 is not aligned by 512 bytes" (it isn't the exact error message, my translation from my localized version, not sure how to change UI language there). RPI Imager v.1.8.5 (latest available).
Very probably something wrong with my settings or steps.

Configuration changes:
custom.txt:
HOSTNAME="pi-cnc"

userdata.txt
NAME="Andrey"
USERNAME="ago"
PASSWORD="*"
VERBOSE="1"
COMPRESS_IMG="1"
ENABLE_COMMIT="1"
COMMIT="342c7ee49e862edc30c893f141f55b9211b7a43b"
KBUSER="ago"
KBHOST="pc-cnc"

After preparing these files manually and ensuring that the board.txt file created 


I also received file .cache/git_fast.linux-rpi-6.1.y
GIT_FAST="true"
GIT_COMMIT="afd5f659b0453e4c710ce5cf74c577563ff16239"

Output files:
rpi-4-debian-bookworm-6.1.77-rt21-arm64-ext4-2024-02-25-2312.img.xz
rpi-4-debian-bookworm-6.1.77-rt21-arm64-ext4-2024-02-25-2312.img.xz.sha256
output/bcm2711/linux-headers-bcm2711-rpi-4_6.1.77-rt21-1_arm64.deb
output/bcm2711/linux-image-bcm2711-rpi-4_6.1.77-rt21-1_arm64.deb
output/logs (attached)

I will be very pleased if you can suggest something to move on.
Thank you!

 

Decided to add some probably useful details :

1. After preparing the userdata.txt file I called "make menu" and went through all the UI (for bcm2711 arm64) till the end.

2. I used branch linuxcnc-2.9.2 of rpi-img-builder-lcnc according to Rod's suggestion.

3. I used default image size IMGSIZE="122880MB"

Got it via balenaEtcher!
Thanks guys!

I still to customize it's vnc-server as I use headless CNC and install LinuxCNC itself. I didn't find it in the installed programs...
Will dig your framework deeper how to customize adding of the additional sources and gpg keys (to have linuxcnc, vscode, and sublime on the board).

Again - great work - The building process took about 30 minutes on 16 cores!

Hello guys, I continue experiments with just created image.
I have burned it on the SD Card 512Gb (not sure, probably it is to much?)
Did you faced with troubles like read-only root file system?
Here is what I have checked:

ago@pi-cnc:~$ sudo mount | grep /dev
udev on /dev type devtmpfs (rw,nosuid,relatime,size=1667364k,nr_inodes=416841,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
/dev/mmcblk0p2 on / type ext4 (ro,noatime,errors=remount-ro,commit=600)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
/dev/mmcblk0p1 on /boot/broadcom type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,flush,errors=remount-ro)

ago@pi-cnc:~$ sudo dmesg | grep mmcblk0p2
[    4.946191] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Quota mode: none.
[    7.221878] EXT4-fs (mmcblk0p2): re-mounted. Quota mode: none.
[  216.642005] EXT4-fs error (device mmcblk0p2): ext4_validate_block_bitmap:420: comm http: bg 128: bad block bitmap checksum
[  216.642025] Aborting journal on device mmcblk0p2-8.
[  216.722933] EXT4-fs error (device mmcblk0p2): ext4_journal_check_start:83: comm kworker/u8:0: Detected aborted journal
[  217.348056] EXT4-fs (mmcblk0p2): Remounting filesystem read-only
[  217.348076] EXT4-fs error (device mmcblk0p2) in ext4_writepages:2856: IO failure
[  217.348079] EXT4-fs (mmcblk0p2): failed to convert unwritten extents to written extents -- potential data loss!  (inode 108110, error -30)
[  217.348634] EXT4-fs (mmcblk0p2): failed to convert unwritten extents to written extents -- potential data loss!  (inode 108110, error -30)
[  217.348963] EXT4-fs (mmcblk0p2): failed to convert unwritten extents to written extents -- potential data loss!  (inode 108114, error -30)
[  217.349014] EXT4-fs (mmcblk0p2): failed to convert unwritten extents to written extents -- potential data loss!  (inode 108116, error -30)
[  217.349153] EXT4-fs (mmcblk0p2): failed to convert unwritten extents to written extents -- potential data loss!  (inode 108117, error -30)
[  217.349325] EXT4-fs (mmcblk0p2): failed to convert unwritten extents to written extents -- potential data loss!  (inode 108111, error -30)
[  217.349366] EXT4-fs (mmcblk0p2): failed to convert unwritten extents to written extents -- potential data loss!  (inode 108113, error -30)
[  217.349455] EXT4-fs (mmcblk0p2): failed to convert unwritten extents to written extents -- potential data loss!  (inode 108115, error -30)
[  217.349507] EXT4-fs (mmcblk0p2): failed to convert unwritten extents to written extents -- potential data loss!  (inode 108112, error -30)
[ 1171.143967] EXT4-fs error (device mmcblk0p2): __ext4_remount:6425: comm mount: Abort forced by user

Thank you for your help!

I suspect your SSD is too big given the bugs in the volume sizing upstream. They never imagined people would use large volumes or build images as big as we need. I have not merged upstream changes which might fix it. I will try and do that.

If you want to add extra packages, the best method I found was to use Debian PAckage search packages.debian.org/index to locate your exact package and then add to this line
github.com/rodw-au/rpi-img-builder-lcnc/...ata.txt#L24C1-L24C11

Anything else needs to be done in  userscripts
github.com/rodw-au/rpi-img-builder-lcnc/...userscripts/uscripts
There are several section in  this file called at different times  but most of it is done in  run_function2 when the chroot exists
Files can be copied out of the chroot in run_function3  but the file paths change.
 

Is it a known working SD card at that capacity ?
A bit of searching for that particular error points to a file system error, you may have to run fsck on the SD card on a Linux system.
I doubt that it has anything to do with the size of the drive, more likely filesystem error on the SD card.
I do a google search for the text contained in the error.

I read what is reported in github.com/golyakoff/linux-info/blob/mai...aspberryOS/README.md.
My suggestion is to add (as reported here ubuntu.com/blog/industrial-embedded-systems-ii and here docs.ros.org/en/foxy/Tutorials/Miscellan...ernel-for-ROS-2.html)

Set CONFIG_HZ_1000 (note: this is no longer in the General Setup menu, go back twice)
-> Processor type and features
-> Timer frequency (1000 HZ)
(X) 1000 HZ

# Set CPU_FREQ_DEFAULT_GOV_PERFORMANCE [=y]
-> Power management and ACPI options
-> CPU Frequency scaling
-> CPU Frequency scaling (CPU_FREQ [=y])
-> Default CPUFreq governor (<choice> [=y])
(X) performance

Any kernel parameters can be set here
github.com/rodw-au/rpi-img-builder-lcnc/...rscripts/uscripts#L6
Any kernel compile time settings can be set in menuconfig and then the  defconfig file can be saved for future compiles.

Thank you cornholio and rodw for your answers!
- About the SD Card and I decided to think that it was some troubles with SD partition and just taken another one, if it will be reproduced - I'll dig it deeper.

- I understand the idea of selecting kernel-and-patch consistency, in my actual test I try to take fresh kernel 6.1.77 and corresponding RT patch 77-rt24.

I also excited about the way of installing additional packages from standard Debian sources as well as from additional using userscripts.

rodw, I also would suggest a small fix to support the spaces in full name, otherwise someone will receive something like that (example for full name "Andrey CNC"):
+ adduser ago --gecos Andrey CNC --disabled-password
adduser: The user `ago' does not exist.

As I understand it is placed in two places:
* github.com/rodw-au/rpi-img-builder-lcnc/...ccount#L12C29-L12C36
* github.com/rodw-au/rpi-img-builder-lcnc/.../scripts/stage2#L112
(Not sure about the correct fix, I am not very experienced with bash yet. I tried to fix it with '${NAME}' but just received name ${NAME}
I am very excited to the way you did this image builder!

cornholio, you said that you did something similar to my experiment with the original Raspberry Pi OS image. I slightly confused with my tutorial at the early beginning of this thread. Nevertheless I did it once, I can not repeat it anymore... Not sure what the problem is, unfortunately screen shows very fast several red lines in the console during the boot and then I see the only black screen. Did you posted your steps somewhere? I I will be very pleased if you could briefly take a look at my solution and find problems there if any.