Virtual Machines#
Virtual machines allow running a guest operating system over a host - so you could run a different Linux distro on a certain host distro or Linux on a Mac, for example.
They do this by emulation of the guest architecture over the host, either fully in software or where feasible via hardware acceleration. The latter, where available, is much more performant.
Working on Mac#
Given that we mostly work on Mac with Arm chips we will often need virtualization to run Linux x86 or arm emulated systems. This is particularly relevant in the case of running containers.
VMs or Containers#
Containers are more lightweight than VMs and are easier to run in workflows and automations. It is easier to specify a container configuration than a VM.
For these reasons generally you should prefer working with containers unless you specifically need to work with a full blown VM.
UTM#
UTM is a graphical application for working with VMs on Mac, built on the lower level QEMU (described below). You can install it as a brew cask:
brew install utm
You can follow the UI guide to install a downloaded ISO of the operating system you are hoping to set up. When installing the OS you will likely want a server/minimal installation rather than installing graphical packages. If your distro has an ‘ssh server’ option that is a good choice.
You are likely to want to SSH into the VM after it is set up, to do so in UTM you need to edit the VM image in UTM, and under ‘Network’ change the ‘Network Mode’ to Emulated VLAN. There you can set up port forwarind so you can SSH to a non-standard port from the host.
You may want to ‘clone’ a cleanly set up VM so you have something to revert to if needed. Note, this will use a significant amount of disk space but is currently the only straightfoward approach with UTM on MacOS. The are further options with QEMU snapshots, described later, if working at that level.
QEMU#
QEMU (Quick Emulator) is free software for emulating computers, i.e. running virtual machines. It is often used as a basis for virtualization technologies on Linux and Mac, for example in Docker or Podman machines on Mac, or in UTM which provides a UI on top for Mac.
If you are more comfortable working in a shell than UI or are thinknig about automation it is useful to start with QEMU directly for creating and running VMs. (Note: there are many tools for managing the VMs at a higher level, like libvrt - but it is good to start with the lower-level tool first).
Creating a VM#
You will need to start with an ISO image - which you can download from your chosen operating system (or distro’s) web page. The systems team have some stored on USB if you are in the Dublin office also.
Install qemu, on Mac it is:
brew install qemu
It may be already installed if you have Docker/Podman running. Create a blank VM image, giving plenty of space to install everything - here 15 GB. Give it a descriptive name, e.g. ‘distro_version_arch.raw’.
qemu-img create -f raw my_vm_image.raw 15G
Now we will boot from the ISO and install the system on the blank image:
qemu-system-x86_64 -cdrom my_distro_iso.iso -boot order=d -drive file=my_vm_image.raw,format=raw -m 4G -smp 4
Here are giving 4GB RAM and letting it see 4 CPUs. Note that here we are running an x86_64 version without hardware acceleration, which will be slow on Mac M1. There are ways to run ARM versions of qemu-system on Mac which involve fiddling with more options - you can use a UTM config as a reference for this if interested.
Distro Installtion#
The above command will launch the graphical installer for your distro. You can choose a full ‘UI’ install, but your VM will be smaller and lighter if you choose a more minimal installation - e.g. ‘Server’ type on RHEL-like distros. This is particularly important on Mac Arm machines - since QEMU can only do software emulation here you will need to be fairly minimimal with what is running.
For RHEL-like distros during installation you can add a root password and normal user with admin rights. Don’t forget to enable the Ethernet connection in the Networks section.
If on Mac the installtion could take a long time, since QEMU is doing software emulation only. When finished shut down the VM.
Setting up SSH#
If eventually running the VM headless, which is recommended for efficient resource use you will want to set up an SSH server and expose its port to the host.
Boot back into the VM with:
qemu-system-x86_64 my_vm_image.raw -m 4G -smp 4 -nic user,hostfwd=tcp::8080-:22
The nic arguments will allow us to ssh into the machine. You will be greated by a barebones terminal which you can log into, run an update (could take some time) and start the ssh service (assuming RHEL-like):
sudo dnf update
sudo systemctl enable sshd
You should now be able to ssh into the VM from the host:
ssh $my_user@127.0.0.1 -p 8080
Finally, we can shutdown the VM (shutdown now in terminal) and add the following to a script start_vm.sh to run it headless from now on:
echo 'qemu-system-x86_64 my_vm_image.raw -m 4G -smp 4 -nic user,hostfwd=tcp::8080-:22 -display none' > start_vm.sh
Transferring files#
After setting up ssh you can use scp to transfer files (or use rsync or IDE integrations for fancier things):
scp -P 8080 -r my_dir 127.0.0.1:/home/my_user