Gaming on Linux with a Windows Virtual Machine using OVMF

Back in 2014, I made a small guide on KVM virtualization that showed how VFIO-VGA could be used to pass a video card to a virtual machine from the virtualization host. I've only used Windows in virtual machines since that time. I've also transitioned from Nvidia to AMD for the host's graphic card to get better Linux support. This meant that I had to rebuild my gaming virtual machine using Open Virtual Machine Firmware (OVMF) instead of the standard BIOS.

kvm_with_ovmf

Image - Windows 7 with dedicated Nvidia Geforce using KVM/QEMU and OVMF

At the time, the functionality was bleeding edge in QEMU and in the Linux kernel. Nowadays, even Debian Stretch has most of the features out of the box and most of the work can be done through a graphic user interface. The following sections regroup most of my notes on the subject and can be used like a form of guide. The improvements over the previous guide are the following:

  • Integration with other libvirt tools such as virsh and virt-manager
  • No more custom kernel compilation
  • Reliable guest virtual machines
  • Stable operating system (like it or not, Debian Sid tends to break sometimes)
  • Video card and USB initialization happens before the guest OS boots

Requirements

Hardware requirements remain the same from the previous guide. Your platform will have to support:

  • Virtualization extensions and IOMMU, either through Intel VT-x and VT-d or AMD-V and AMD-Vi. Also make sure that your motherboard supports those features.
  • You will also need to have at least 2 graphics adapters (one for the host and another for the guest) - it is easier to have two separate cards but there are ways to use the onboard graphics if you are in that situation.

For software, I am using Debian Stretch but this should also work for recent Ubuntu releases. There are also many guides for other Linux distribution.

Initial setup from fresh Debian installation

Essentially, the guide from the Debian wiki can be followed to setup a brand new installation of Debian for basic virtualization.

# install the necessary packages
apt-get update
apt-get install qemu-kvm libvirt-clients libvirt-daemon-system ovmf

# add your user to the libvirt management group
adduser <youruser> libvirt
adduser <youruser> libvirt-qemu

# Log out and back to test your user access to the system virtualization
virsh --connect qemu:///system list --all

Once your virtualization host is setup, you will be able to use the Virtual Machine Manager (virt-manager) graphic interface to manage your virtual machines.

Configure IOMMU and VGA passthrough options

Configure your virtualization host by creating a new configuration file for modprobe. Here is a copy of my file:

# /etc/modprobe.d/kvm.conf

# Set virtualization options (depends on your platform)
options kvm ignore_msrs=1
options kvm_intel emulate_invalid_guest_state=0

# Prevent loading NVIDIA graphics (ensure guest video card is not used)
blacklist nouveau
options nouveau modeset=0

# # If your motherboard does not support interrupt remapping
# # Watch your logs closely with this option
options vfio_iommu_type1 allow_unsafe_interrupts=1

You need to find the GPU device ID by using the lspci command and searching for your graphic card and its associated audio interface. I am using an Nvidia graphics card at the moment (10de:1c82 and 10de:0fb9):

lspci -nn | grep -i nvidia
02:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP107 [GeForce GTX 1050 Ti] [10de:1c82] (rev a1)
02:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:0fb9] (rev a1)

Now there are many ways to isolate the GPU, I have had a good experience with pci-stub but there are other methods (see the Arch Linux wiki). I am using pci-stub right from the boot command line to ensure that the card is not used by anything else during the system startup.

# /etc/default/grub

# ...
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash iommu=on intel_iommu=on pci-stub.ids=0000:10de:1c82,0000:10de:0fb9"
# ...

Now update your boot loader and your initramfs and restart the system:

update-initramfs -u -k all
update-grub
reboot

If it works, you should be able to see messages in dmesg about IOMMU and your devices that are claimed by pci-stub.

Creating Windows virtual machine

Using the Virtual Machine Manager, it is easy to create a new virtual machine. After the operating system is installed, we will be attaching the graphic card and a USB controller to the VM.

To start the process, we can create a new virtual machine from the Virtual Machine Manager normally. At the end of the creation wizard, don't forget to check the 'Customize configuration before install' checkbox.

vm_01

We have to make a few adjustments to ensure that the VM has the Windows virtio drivers (direct link to the latest ISO). Add a new IDE CDROM and connect to the drivers disk and change the IDE disk bus from IDE to VirtIO for better performance.

vm_02

Finally, make sure to select the UEFI firmware type in the 'Overview' section to make use of OVMF. After that, save everything and the VM should boot from the Windows disk.

vm_03

During installation, choose the Custom (advanced) installation type and use the Load Driver link to install the storage drivers for your Windows version. Mine were in E:\viostor\w7\amd64\.

vm_04

From there, you can finish installing Windows normally. Once that is done, install the remaining VirtIO drivers from the Windows Device Manager by letting the operating system find the proper drivers on the VirtIO drivers disk.

Attaching the graphics card to the virtual machine

Once your operating system is installed with the VirtIO drivers, you can shut down the virtual machine. At this point, you will be able to attach a PCI Host Device. Select your graphics card from the device list and also attach your GPU audio adapter to allow the card to the passed to the guest.

vm_04

If you are using an Nvidia Geforce card for the guest virtual machine, you will probably encounter a code 43 error. This is because Nvidia drivers will detect the virtual machine and return the error. To work around this, you will have to make some adjustments to your virtual machine XML domain definition. The easiest way to do so is by using virsh --connect qemu:///system list --all and virsh --connect qemu:///system edit vmname. From there, you will need to add an hv_vendor_id Hyper-V feature and also hide KVM from the guest.

<!-- ... -->
<features>
    <hyperv>
        <vendor_id state='on' value='NvidiaIsBad'/>
    </hyperv>
    <!-- ... -->
    <kvm>
        <hidden state='on'/>
    </kvm>
</features>
<!-- ... -->

From there, you can save the definition file and exit the editor. You should be able to remove the Spice and Tablet devices from the Virtual Machine Manager. Save everything and start up the VM. You should now see your VM boot using the graphic cards.

Connecting the virtual machine to PulseAudio

Because you have removed the Spice devices from the virtual machine, you will not have audio unless you have passed a dedicated sound card to the guest. This is perfectly normal. Most desktop environments are using PulseAudio (PA) for the sound stack.

It is possible to load a TCP server module for PA that will allow forwarding of the guest audio emulated device to the host through libvirt.

This module has to be explicitely enabled by adding the module-native-protocol-tcp and only allowing connections from 127.0.0.1 address. The auth-anonymous=1 is required because libvirt can not provide the cookie that PA expects for authentication. For additional security, you can use a firewall to drop any incoming requests to the PA TCP service.

# /etc/pulse/default.pa
load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1 auth-anonymous=1

Log out and back again to reload your PA profile.

Once PA is configured, you will need to edit the libvirt XML domain again (using virsh edit).

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
    <!-- ... -->
    </devices>
    <qemu:commandline>
        <qemu:env name='QEMU_AUDIO_DRV' value='pa'/>
        <qemu:env name='QEMU_PA_SERVER' value='127.0.0.1'/>
    </qemu:commandline>
</domain>

From there, you should have audio from your virtual machine.

Performance tweaks

Once the configuration is completed and all the drivers are installed, you should be able to start performing a few performance enhancements. The most common performance tweaks are known as Hyper-V enlightenment. These tweaks might not always work depending on your guest operating system version. I had problems with them on Windows 7 with OVMF but they should work with Windows 10.

Other tweaks to the virtual machine can be made by using CPU pinning, use of huge pages, improved block device I/O caching. However, most of these tweaks will depend on your setup.

Additional references: