Enabling Hardware Passthrough on Proxmox
I won't lie, this post is in large part for my own benefit. I set up a lot of Proxmox systems, and there are a few steps that need to happen for hardware passthrough to work correctly. This is the quick and dirty guide to get it working.
1) First pop into the BIOS and enable anything and everything related to:
- VT-d or "Virtualization"
- SR-IOV
- x2APIC or APIC
2) Now edit /etc/modules and add the following:
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
3a) If your boot drive is a regular disk (not a ZFS mirror)
...your bootloader will be GRUB. Go ahead and edit /etc/default/grub
Modify the GRUB_CMDLINE_LINUX_DEFAULT
line and add intel_iommu=on
(if on an Intel CPU), amd_iommu=on
(if on an AMD CPU) and iommu=pt
for both. It should look something like this:
GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"
GRUB_CMDLINE_LINUX=""
Save that file, and update grub:
update-grub
You will need to reboot to apply the changes.
3b) If your boot drive is a ZFS Mirror (not a regular disk)
...you'll be using systemd-boot (not grub)
Edit /etc/kernel/cmdline
Modify the root=ZFS=rpool
line and add intel_iommu=on
(if on an Intel CPU), amd_iommu=on
(if on an AMD CPU) and iommu=pt
for both. It should look something like this:
root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet intel_iommu=on iommu=pt
Now refresh the bootloader:
proxmox-boot-tool refresh
You will need to reboot to apply the changes.
That's it! Great Job! Passing through PCIe devices in Proxmox will (probably) work.
There are some cases where this won't work, so if it fails, take a peek at the official documentation here: https://pve.proxmox.com/wiki/PCI(e)_Passthrough