I got access to a StarFive VisionFive 2 and wanted to play around with Real-Time Linux on it.
This article goes into excruciating detail on how I set that machine up to with PREEMPT_RT
.
The first mainline Kernel with PREEMPT_RT is v6.12. At the time of writing v6.15 is the newest kernel and my Debian laptop runs on v6.1.0. For the Debian user that I am v6.12 is pretty new.
So, how do we get this single board computer (SBC) to work and run on such a new kernel? There is no StarFive Debian release with a Linux kernel v6.12 or newer. So we need to make due with engineering release 202409 and kernel v6.6.20 and get to v6.12 later. (That's still newer than my Debian laptop...)
I just stick to the official installation guide for Debian on the VisionFive 2. I chose to boot off of an SD card, pop it in my laptop and run:
bzip2 -d starfive-jh7110-202409-SD-minimal-desktop-wayland.img.bz2sudo dd if=starfive-jh7110-202409-SD-minimal-desktop-wayland.img of=/dev/sda conv=fsync bs=4M status=progress
Now we can insert the SD card into the SBC and watch it boot for the first time.
I have a USB to UART adapter attached to my laptop and the SBC. After having added myself to the dialout group on my laptop I rebooted (a relogin would have done it as well):
sudo usermod -a -G dialout chrissudo reboot
Now I plug in the adapter to my laptop and check what tty adapter it is on:
sudo dmesg
This should print something including the tty path. I use this path to attach a terminal to that adapter.
screen -U /dev/ttyUSB0 -b 115200
Exit with Ctrl+a d
.
There is this magic of LEDs on a UART adapter blinking as you type.
Linux is running on the SBC but I'd like to have network access. I have my Linux Laptop right here and want to share its network connection with the SBC.
First we need to know what network cards we're dealing with:
~ λ ip link1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:002: wlp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DORMANT group default qlen 1000 link/ether 10:6f:d9:1e:4f:e1 brd ff:ff:ff:ff:ff:ff3: enx00133bb15f5b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000 link/ether 00:13:3b:b1:5f:5b brd ff:ff:ff:ff:ff:ff```
As you can see I have three network cards, the loopback device (i.e. for localhost) my wifi card wlp1s0
and a usb LAN adapter enx00133bb15f5b
.
We need to setup NAT to forward packets meant for the SBC with the laptop.
sudo sysctl -w net.ipv4.ip_forward=1sudo iptables -t nat -A POSTROUTING -o wlp1s0 -j MASQUERADEsudo iptables -A FORWARD -i enx00133bb15f5b -o wlp1s0 -j ACCEPTsudo iptables -A FORWARD -i wlp1s0 -o enx00133bb15f5b -m state --state RELATED,ESTABLISHED -j ACCEPT
We want to assign a new IP address for the laptop to the enx00133bb15f5b
link.
sudo ip addr add 192.168.10.1/24 dev enx00133bb15f5bsudo ip link set enx00133bb15f5b up
I check that this worked with:
~ λ ip addr1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host noprefixroute valid_lft forever preferred_lft forever2: wlp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether 10:6f:d9:1e:4f:e1 brd ff:ff:ff:ff:ff:ff inet 172.17.65.91/17 brd 172.17.127.255 scope global dynamic noprefixroute wlp1s0 valid_lft 1694sec preferred_lft 1694sec inet6 2a00:1398:9:fb03:c786:4d2:ce2e:e30d/64 scope global dynamic noprefixroute valid_lft 2591999sec preferred_lft 604799sec inet6 fe80::cebf:38eb:9589:c3a0/64 scope link noprefixroute valid_lft forever preferred_lft forever3: enx00133bb15f5b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 00:13:3b:b1:5f:5b brd ff:ff:ff:ff:ff:ff inet 192.168.10.1/24 scope global enx00133bb15f5b valid_lft forever preferred_lft forever
The enx00133bb15f5b
link has a new IP address (192.168.10.1) assigned to it.
user@starfive:~$ ip link1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:002: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 link/ether 6c:cf:39:00:49:50 brd ff:ff:ff:ff:ff:ff altname end03: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 link/ether 6c:cf:39:00:49:51 brd ff:ff:ff:ff:ff:ff altname end14: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000 link/sit 0.0.0.0 brd 0.0.0.0
I have attached the LAN cable to the eth0
network card.
sudo ip addr add 192.168.10.2/24 dev eth0sudo ip link set eth0 upsudo ip route add default via 192.168.10.1
With this we have added the ip 192.168.10.2
to the SBC's eth0
interface.
DHCP would have done that automatically but we didn't setup a DHCP server on the laptop so we have to do this manually.
sudo vi /etc/resolv.conf
Add this line (we all like Google, don't we?):
nameserver 8.8.8.8
We don't need the NetworkManager for this, btw. Feel free to disable it:
sudo systemctl disable --now NetworkManager
# ping the laptop# this tests the link worksping 192.168.10.1# ping google infrastructure without the DNS# this tests the NAT worksping 8.8.8.8# ping google with DNS# this tests that the DNS worksping google.com
This setup works most of the time. Sometimes it's a bit flaky. I presume that's because the NetworkManager on my laptop interferes but I'm not sure and can't turn it off there. I often have to run the commands multiple times until they stick.
There already is an SSH server installed and running on the SBC. All I have to do is connect to it from the host:
ssh user@192.168.10.2
So much faster than UART, damn!
No blinking lights, though :<
Again we refer to the VisionFive 2 docs.
sudo apt-get -y install build-essential linux-source bc kmod cpio flex libncurses5-dev libelf-dev libssl-dev dwarves bison git debhelpergit clone --depth 1 --single-branch --branch JH7110_VisionFive2_6.12.y_devel https://github.com/starfive-tech/linuxcd linuxcp arch/riscv/configs/starfive_visionfive2_defconfig .configmake ARCH=riscv olddefconfig
Now we need to configure PREEMPT_RT:
make ARCH=riscv menuconfig
In this menu screen we select Scheduler controlled preemption model mode under General setup then Preemption Model.
In my case I also had to set this in the .config
file in order to differentiate this kernel from a PREEMPT_DYNAMIC
build I also created.
CONFIG_LOCALVERSION="preempt-rt"
Back to the documentation and we start the compilation:
make ARCH=riscv -j$(nproc) bindeb-pkg# for me the .deb files ended up being in the $HOME directorycd ..sudo dpkg -i \ linux-headers-6.12.5*_riscv64.deb \ linux-image-6.12.5*_riscv64.deb \ linux-libc-dev_6.12.5*_riscv64.deb
When doing so more than once, I had to uninstall an old kernel; probably because there isn't enough space for the rgx firmware on initrd but I'm really not sure.
This takes a while — some time to do the toilet, nice. Still not done? Then I have some time to write this article, also nice. Maybe I should've crosscompiled on my laptop. Compiling the Linux kernel on a computer that doesn't even have a heat-sink is something... That took roughly an hour.
Some notes:
The documentation warns about the dtbs not being synced but for me they were, so I left that as is and rebooted.
Installing debhelper
was missing in the documentation.
I ended up having two v6.12.5 kernels installed: a PREEMPT_RT
and a PREEMPT_DYNAMIC
one.
To choose which one to boot I edited /boot/extlinux/extlinux.conf
.
It has such a nice warning not to edit it.
Here I just renamed the l0
and l0r
labels to l1
and l1r
and the other way around.
I also rearranged the labels to be a little nicer; I have no idea if that was needed.
I did try using a proper method of changing the kernel to boot but this was the only thing I came up with and it works.
I left the l2
and l2r
labels in case I screwed something up.
sudo vi /boot/extlinux/extlinux.conf
## /extlinux/extlinux.conf
##
## IMPORTANT WARNING
##
## The configuration of this file is generated automatically.
## Do not edit this file manually, use: u-boot-update
default l0
menu title U-Boot menu
prompt 0
timeout 50
label l0
menu label Debian GNU/Linux trixie/sid 6.12.5preempt-rt+
linux /vmlinuz-6.12.5preempt-rt+
initrd /initrd.img-6.12.5preempt-rt+
fdtdir /dtbs/6.12.5preempt-rt+
append root=/dev/mmcblk1p4 root=/dev/mmcblk1p4 rw console=tty0 console=ttyS0,115200 earlycon rootwait stmmaceth=chain_mode:1 selinux=0
label l0r
menu label Debian GNU/Linux trixie/sid 6.12.5preempt-rt+ (rescue target)
linux /vmlinuz-6.12.5preempt-rt+
initrd /initrd.img-6.12.5preempt-rt+
fdtdir /dtbs/6.12.5preempt-rt+
append root=/dev/mmcblk1p4 root=/dev/mmcblk1p4 rw console=tty0 console=ttyS0,115200 earlycon rootwait stmmaceth=chain_mode:1 selinux=0 single
label l1
menu label Debian GNU/Linux trixie/sid 6.12.5+
linux /vmlinuz-6.12.5+
initrd /initrd.img-6.12.5+
fdtdir /dtbs/6.12.5+
append root=/dev/mmcblk1p4 root=/dev/mmcblk1p4 rw console=tty0 console=ttyS0,115200 earlycon rootwait stmmaceth=chain_mode:1 selinux=0
label l1r
menu label Debian GNU/Linux trixie/sid 6.12.5+ (rescue target)
linux /vmlinuz-6.12.5+
initrd /initrd.img-6.12.5+
fdtdir /dtbs/6.12.5+
append root=/dev/mmcblk1p4 root=/dev/mmcblk1p4 rw console=tty0 console=ttyS0,115200 earlycon rootwait stmmaceth=chain_mode:1 selinux=0 single
label l2
menu label Debian GNU/Linux trixie/sid 6.6.20-starfive
linux /vmlinuz-6.6.20-starfive
initrd /initrd.img-6.6.20-starfive
fdtdir /dtbs/6.6.20-starfive
append root=/dev/mmcblk1p4 root=/dev/mmcblk1p4 rw console=tty0 console=ttyS0,115200 earlycon rootwait stmmaceth=chain_mode:1 selinux=0
label l2r
menu label Debian GNU/Linux trixie/sid 6.6.20-starfive (rescue target)
linux /vmlinuz-6.6.20-starfive
initrd /initrd.img-6.6.20-starfive
fdtdir /dtbs/6.6.20-starfive
append root=/dev/mmcblk1p4 root=/dev/mmcblk1p4 rw console=tty0 console=ttyS0,115200 earlycon rootwait stmmaceth=chain_mode:1 selinux=0 single
Now we can do some testing of our real-time performance with cyclictest:
sudo apt install rt-tests stress-ngsudo echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor# cpustress-ng --matrix 4# memstress-ng --vm 4sudo cyclictest --mlockall --smp --priority=99 --interval=200 --quiet --duration=10m --histofall=200 > out.txt
CONFIG_NO_HZ_IDLE
CONFIG_HZ_1000
instead of CONFIG_HZ_100
To get my custom configs to work I had to edit my .bash_profile
: unix.stackexchange.com/questions/94490/bash-doesnt-read-bashrc-unless-manually-started