Thursday, July 19, 2012

HowTo - KVM Passthrough

1. Install Qemu-KVM [1]

1.1 Enable the virtualization setting in the BIOS first.

1.2 Check KVM installation environment.
$ sudo apt-get update

$ sudo apt-get install cpu-checker

$ sudo modprobe kvm_intel

$ sudo modprobe kvm
$ kvm-ok

1.3 If everything is okay, you can install the qemu-kvm packages.
$ sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils

$ sudo adduser `id -un` libvirtd

$ sudo chmod 777 /var/run/libvirt/libvirt-sock

$ sudo chown root:libvirtd /dev/kvm

$ virsh -c qemu:///system list

1.4 If the command below shows you an empty list of virtual machines, the installation is successful. You may now install the virt-manager. which is the GUI tool for KVM.
$ sudo apt-get install virt-manager

2. Enable IOMMU support

$ sudo vi /etc/default/grub
  • Add intel_iommu=on in GRUB_CMDLINE_LINUX_DEFAULT parameter.
$ sudo update-grub

$ sudo reboot

You can verify the IOMMU support by type in
$ dmesg | grep -e DMAR -e IOMMU

If you are using an Intel machine. For AMD, check here.

3. Modify KVM kernel [2]

I skipped this step.

4. Verify the PCI that you need to attach to the virtual machine.

$ lspci

Suppose we are interested in passing through this device
20:00.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI Caicos [Radeon HD 6450]

which is a PCI-Express Graphics Card. This PCI device 20:00.0 will be the example device in the following steps.

$ lspci -n
...
20:00.0 0300: 1002:6779
...

Note down the device ID 1002:6779 .

5. Unbind and bind

5.1 Load the PCI stub module.
$ sudo modprobe pci_stub
5.2 Unbind the device from the host kernel driver, and bind it with pci stub.
# echo "1002 6779" > /sys/bus/pci/drivers/pci-stub/new_id
# echo 0000:20:00.0 > /sys/bus/pci/devices/0000\:20\:00.0/driver/unbind
# echo 0000:20:00.0 > /sys/bus/pci/drivers/pci-stub/bind

6. Assign device
Add extra parameters as below after the usual kvm command.

$ kvm ... -device pci-assign,host=20:00.0

If you see errors, check the following section.

7. Necessary permission modifications [3]

According to my experience, you might encounter an error saying:
Device 'pci-assign' could not be initialized.

Before you try any solution, first make sure that modules such as kvm, kvm_intel, pci_stub are loaded. Also make sure that intel_iommu=on is in your grub configuration file (even if you see IOMMU from dmesg, it doesn't ensure that INTEL_IOMMU is enabled).

The command that fixed it was:
# echo 1 > /sys/module/kvm/parameters/allow_unsafe_assigned_interrupts

But I did try something else before the command above, so I suppose that if this command doesn't work, try the following:
# vi  /etc/libvirt/qemu.conf

Uncomment and set user and group to "root".
Set clear_emulator_capabilites = 0 .
Set relaxed_acs_check = 1 .

Besides, if you are using a RPM-based system such as Fedora, you also need to set SELinux to  permissive.

Then run kvm under root.
# kvm ... -device pci-assign,host=20:00.0



8. Done.

In the guest OS, run lspci, and check if you can see the assigned pci device there.


[1] https://help.ubuntu.com/community/KVM/Installation
[2] http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM
[3] http://comments.gmane.org/gmane.comp.emulators.libvirt.user/2945

5 comments:

  1. I was struggling with this. Your post saved my day!

    ReplyDelete
  2. This comment has been removed by the author.

    ReplyDelete
  3. internal error process exited while connecting to monitor: Failed to assign device "hostdev0" : Permission denied
    2016-04-09T05:52:40.025077Z qemu-kvm: -device pci-assign,host=01:00.0,id=hostdev0,configfd=27,bus=pci.0,addr=0x4: Device 'pci-assign' could not be initialized could u give me some advice

    ReplyDelete
  4. Hi,

    I am getting the following error.

    libvirt: QEMU Driver error : internal error: process exited while connecting to monitor: 2017-08-08T07:03:30.225716Z qemu-kvm: -device pci-assign,configfd=36,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x5: Failed to assign device "hostdev0": Invalid argument

    Can u tell me what could be the problem ?

    ReplyDelete