Installing Linuxcnc and Debian Bookworm on problematic hardware (eg. Realtek NIC

More
16 Dec 2022 10:21 #259620 by rodw
Running Debian Bookworm or Bullseye with some hardware (notably PC’s with Realtek network interfaces) has proved problematic as a new type of latency emerged. Network Latency between the PC and motion hardware. If the network becomes too slow, there is insufficient time for Linuxcnc to get things done and mesa network hardware shuts down after issuing an error finishing read. Ethercat drives with slaves disappearing intermittently are also likely to be caused by this issue. Previously stable hardware has become problematic due to this problem.The issue occurs on some hardware with any version of debian from kernel 5.10 (Bullseye) and above through to the latest kernel 6.1.

These are the versions of Debian to support modern hardware. Installing Bookworm is an attractive method to get Linuxcnc installed as it is deployed as a Debian package on this Debian version.

There are two issues involved here.
1. A problematic PREEMPT_RT realtime kernel deployed with Debian that adds to network latency.
2. A confusing set of realtek network drivers in Debian and a mismatch between the default driver and the hardware.
Resolving the first issue used to require compiling the Linux kernel but now a prebuilt .deb of the kernel and the headers. These have been built using the pristine source of the 6.1 kernel from kernel.org so it's not affected by Debian’s kernel patches.

Realtek drivers.
Debian has 3 Realtek drivers in Debian:
The default r8169 driver used with all realtek hardware by default
The r8168-dkms driver that covers a large range of NIC’s (ref: packages.debian.org/bookworm/r8168-dkms)
The r8125-dkms driver that supports the new 2.5 gb NIC’s found on the Odroid H2+/H3/H3+ pcs and an increasing number of other PC’s (ref: packages.debian.org/bookworm/r8125-dkms)

dkms drivers are compiled from source on installation and need the linux-headers. These contain all the C header files to allow building software against the kernel. The default r8169 has no installable driver. To reinstall it, you need to purge any dkms drivers you have installed (eg sudo apt purge r81XX-dkms)

So it sounds pretty simple right? We just need to install our custom kernel which is likely to be cdimage.debian.org/cdimage/unofficial/no...mware/weekly-builds/ for any PC that does not have Realtek drivers) and install the correct driver for a Realtek NIC.

Note that it's not necessary to reinstall Debian if you are afflicted with this problem, you can just update to our custom 6.1 kernel and its headers on the link below and optionally install the correct Realtek driver.

The next post will work through a step by step approach on how to install bookworm that contains the linuxcnc packages.

Start by downloading the Bookworm non-free version from the weekly builds here
cdimage.debian.org/cdimage/unofficial/no...mware/weekly-builds/
Usually you will want the amd64 version and I usually select the cd version making sure that the PC is connected to a wired network during installation. Burn it to a USB stick and install it in the usual way but be sure you select the XFCE gui. This is mandatory to ensure xorg elements linuxcnc needs exist.



Then we’ll meet up on the next post and get on with it.
Attachments:
The following user(s) said Thank You: Lcvette, petmakris, nf1z

Please Log in or Create an account to join the conversation.

More
16 Dec 2022 10:23 - 14 Jan 2023 21:40 #259621 by rodw
Lets start:
sudo apt-get install geany ethtool grub-customizer
download linux-headers and linux-image from here
drive.google.com/drive/folders/10uwGg5Rv...LtQ8BZhM3At_gODk16na
cd Downloads
sudo dpkg -i linux-headers(tab)
sudo dpkg -i linux-image(tab)
sudo grub-customizer
Note that pressing (tab) above lets linux autocomplete the line for you to save typing
Make the new 6.1 kernel the default. Eg move to the first entry.
reboot and check our kernel loaded
sudo reboot
debian@debian:~$ uname -v
#4 SMP PREEMPT_RT Wed Dec 14 00:49:27 AEST 2022
debian@debian:~$ uname -r
6.1.0-rt5-linuxcnc-rt
ip a
and save the name of the NIC you will use for your mesa card (enp2s0 for us)
Check the device specs
debian@debian:~$ sudo ethtool -i enp2s0
driver: r8169
version: 6.1.0-rt5-linuxcnc-rt
firmware-version: rtl8168e-3_0.0.4 03/27/12
expansion-rom-version:
bus-info: 0000:02:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no
Note the driver is r8169 but the hardware is rtl8168e
So debian is using the wrong driver. install the correct driver
First check that we are not going to look in CD or DVD drives for software
sudo geany /etc/sources.list
Comment out any lines beginning with CD or DVD with a #
Save and:
sudo apt-get update
sudo apt-get install r8168-dkms
sudo reboot
Once rebooted, open a terminal and:
sudo ethtool -i enp2s0
driver: r8168
note driver is now correctly listed as r8168

So Our job is done! time to configure the wifi at top right and set up the NIC interface for Mesa
But first install linuxcnc
sudo apt-get install linuxcnc-uspace linuxcnc-uspace-dev mesaflash
sudo geany /etc/network/interfaces
Assuming you have already configured the mesa card to be add this text at the end
auto enp2s0
iface enp2s0 inet static
address 10.10.10.1
If not you'll need to set the ip to 192.168.1.1 above,
then use mesaflash to save the 10.10.10.10 ip address to the mesa card first,
then edit the file whats above again so it can be found again.

Restart the network
sudo systemctl restart networking

Open a mesa based configuration (or use pncconf to create one)
Let it run for a while. While it is running,
in another window, check the network latency by reviewing the TMAX settings on your card
halcmd show param hm2*tmax*
Parameters:
Owner   Type  Dir         Value  Name
    36  s32   RW              0  hm2_7i76e.0.read-request.tmax
    36  bit   RO          FALSE  hm2_7i76e.0.read-request.tmax-increased
    36  s32   RW         831032  hm2_7i76e.0.read.tmax
    36  bit   RO          FALSE  hm2_7i76e.0.read.tmax-increased
    36  s32   RW         163024  hm2_7i76e.0.write.tmax
    36  bit   RO          FALSE  hm2_7i76e.0.write.tmax-increased

Note these values are in timer ticks, not nanoseconds.

If something goes wrong and you loose your network interface, reboot into the earlier 6.0 kernel and
the interface should be there. remove it.
sudo apt purge r8168-dkms
reboot into our custom kernel and it should still be there. Re install the driver and reboot the PC to enabl eit.

Follow the tweaks on JT's web site to enable auto login etc
gnipsel.com/linuxcnc/debian-11-fix.html


Also there is a recent issue with Bullseye and Bookworm where it refuses to run linuxcnc on the default gnome desktop which uses wayland. I generally only install the XFCE graphical environment as its a XORG environment and it works perfectly

This can be solved on wayland by adding an environment variable before running linuxcnc
export PYOPENGL_PLATFORM=x11
This may not stick on a reboot. you can add it to your .profile file as follows so its loaded when you log in.
cd ~
nano .profile

add the export statement to the end of the file and save and exit Ctrl-O, Ctrl-X
 
Last edit: 14 Jan 2023 21:40 by rodw.
The following user(s) said Thank You: nf1z

Please Log in or Create an account to join the conversation.

More
16 Dec 2022 10:35 - 16 Dec 2022 10:36 #259623 by rodw
So with that behind you, you should not experience error finidshing read with your mesa card and your ethercat drives should not go AWOL.

Mods might like to sticky  this.
Last edit: 16 Dec 2022 10:36 by rodw.
The following user(s) said Thank You: tommylight, Aciera, spumco

Please Log in or Create an account to join the conversation.

More
16 Dec 2022 11:14 #259625 by tommylight
The following user(s) said Thank You: rodw

Please Log in or Create an account to join the conversation.

More
20 Dec 2022 03:18 #259922 by spumco
Rod,

Thanks for the write-up.  Explains some of the issues I've had with my H2+.

I'm starting a fresh install and, as usual, running in to problems following your process above.  If you don't mind assisting, would you like me to start a new thread or post up here?

thx,
Ralph

Please Log in or Create an account to join the conversation.

More
20 Dec 2022 03:48 #259927 by rodw
I'm happy for you to help :)
By all means post up here as it might help others.
I've been spending a fair bit of time working on a Debian ISO Installer.
Last night I got it to install Debian with my custom kernel from an ISO. Unfortunately it installed it under Bullseye instead of Bookworm so it could not then install Linuxcnc from the Debian repos.

I'm hoping I'll get that fixed in the next day or so. I'm sure I just missed a setting. Using an ISO will solve a lot of problems :)
Worst case, we could install Linuxcnc from pre-built debs.

Please Log in or Create an account to join the conversation.

More
20 Dec 2022 04:01 #259928 by spumco
okdokey...

PC is an Odroid H2+ with the rtl8125 nic.
First hiccup was during installation of Bookworm - additional firmware required despite using the non-free network install iso.

Found the firmware.zip buried somewhere on the Debian site and added it to a USB during install and got past that hurdle.

Your 6.1 rt kernel installed ok.
mill@mill:~$ uname -a
Linux mill 6.1.0-rt5-linuxcnc-rt #4 SMP PREEMPT_RT Wed Dec 14 00:49:27 AEST 2022 x86_64 GNU/Linux

Next issue: it appears I'm unable to install the r8125-dkms driver.
mill@mill:~$ sudo ethtool -i enp3s0
[sudo] password for mill:
driver: r8169
version: 6.1.0-rt5-linuxcnc-rt
firmware-version: rtl8125b-2_0.0.2 07/13/20
expansion-rom-version:
bus-info: 0000:03:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no

error during attempted driver installation

Warning: Spoiler!


LCNC is installed and axis is working.  Haven't bothered to move on the mesaflash yet.

Any ideas?

Please Log in or Create an account to join the conversation.

More
20 Dec 2022 04:18 #259931 by rodw
On the Odroid forum, there is a note that the realtek driver vs the Debian driver is up to 25-30% faster.
I have installed the Realtek Driver on an H2+ but it was back on Buster before it was released in Kernel 5.9. I just had to install from a DVD iso and insert a usb stick containng the realtek source and it worked.

You might need to look at the kernel versions supported by the realtek driver and fall back to a supported kernel version.

You could also try installing the r8125-dkms before installing the kernel. It will rebuild the driver when the kernel is installed via dkms...
Matter of fact one of my pcs righted itself this way....

Hope that helps.

PS I have an Odroid H2+ happilly running Bullseye and Ethercat. :) I'm not going to touch it!

Please Log in or Create an account to join the conversation.

More
20 Dec 2022 04:22 #259932 by rodw
Oops, the Realtek source says its good to kernel 5.19
www.realtek.com/en/component/zoo/categor...pci-express-software

Earlier versions of bullseye ran 5.15, 5.17 and 5.19.
 
The following user(s) said Thank You: spumco

Please Log in or Create an account to join the conversation.

More
20 Dec 2022 04:32 #259934 by spumco
So...  survey says to grease Bookworm, load Bullseye, and get a 5.19 RT kernel?

Please Log in or Create an account to join the conversation.

Time to create page: 0.186 seconds
Powered by Kunena Forum