Support for aarch64 (Raspberry Pi4)

[adaliszk]

Brief description
Since this runner essentially uses QEMU to virtualise other environments, it would be nice if there would be support for ARM, especially for Raspberry Pi. I don't see why would it not work if you anyway virtualise, that itself works.

Steps to reproduce the issue

1. Get a Raspberry Pi4 with Rasbian
2. Set up KVM/QEMU:

sudo apt update
sudo apt full-upgrade
sudo echo "arm_64bit=1" >> /boot/config.txt
sudo apt install libvirt0 binfmt-support qemu-system qemu-user-static
sudo usermod -aG libvirt-qemu $(whoami)
sudo virsh net-start default
sudo virsh net-autostart default

3. Try installing runq:

git clone --recurse-submodules https://github.com/gotoz/runq.git
cd runq
make release
make release-install

Expected behaviour
make does not fail and installs the runtime

Actual behaviour

pi@raspberrypi:~/runq $ make release && make release-install
make -C qemu image
make[1]: Entering directory '/home/pi/runq/qemu'
cd aarch64 && \
docker build -t runq-build-1804 .
/bin/sh: 1: cd: can't cd to aarch64
make[1]: *** [Makefile:6: image] Error 2
make[1]: Leaving directory '/home/pi/runq/qemu'
make: *** [Makefile:26: image] Error 2

Content of section runtimes of /etc/docker/daemon.json
Not yet available as the installation failed:

pi@raspberrypi:~/runq $ cat /etc/docker/daemon.json
cat: /etc/docker/daemon.json: No such file or directory

Content of /var/lib/runq/qemu/proxy --version
Not yet available as the installation failed:

pi@raspberrypi:~/runq $ /var/lib/runq/qemu/proxy --version
-bash: /var/lib/runq/qemu/proxy: No such file or directory

Content of docker --version

pi@raspberrypi:~/runq $ docker --version
Docker version 20.10.6, build 370c289

Additional information

pi@raspberrypi:~/runq $ uname -a
Linux raspberrypi 5.10.17-v8+ #1403 SMP PREEMPT Mon Feb 22 11:37:54 GMT 2021 aarch64 GNU/Linux

pi@raspberrypi:~ $ /usr/bin/qemu-system-aarch64 --version
QEMU emulator version 3.1.0 (Debian 1:3.1+dfsg-8+deb10u8)

pi@raspberrypi:~ $ /usr/bin/qemu-system-x86_64 --version
QEMU emulator version 3.1.0 (Debian 1:3.1+dfsg-8+deb10u8)

[pmorjan]

Currently runq works on x86_64 and s390x. For AArch64 you would have to copy the architecture specific code in ./qemu and in ./cmd/proxy/qemu_<arch>.go. Technically it’s also possible to take Qemu and Linux kernel from the host OS to build the chroot environment for the container instead of copying everything from an Ubuntu base image. But that’s out of scope at the moment. I’m not sure if support for AArch64 it’s worth the effort right now.

But beside all this, from the link to your repo it seems that you want to run x86 Docker images inside Qemu VMs on a Raspberry Pi. That means the Pi would have to emulate x86 hardware for the guest kernel and applications. Is that correct and if so do you have some performance numbers?

[adaliszk]

Hey, yes, the idea is to emulate x86 in some way, and given that your project would make it easy for users is very appealing.

At the moment I'm stuck at trying to emulate with Qemu, I was unable to assign enough RAM to the VM in order to test my image inside it, so I was trying to switch to XEN. I have data about the performance on x86, as soon as I'm able to emulate for the image I would able to compare performance.

With this particular server, I don't expect any major problems as the bottleneck in it's case is Memory and Storage, CPU usage is pretty low.