A couple of things regarding system specs before I begin.

I am currently attempting to pass a Nvidia RTX 3050 GPU through to a Windows 10 VM. I am aiming to use Looking Glass to do this as I am working on translating Japanese games that do not run in Linux into English, and need to look at the game while looking at game scripts to properly do this. My specs are as follows:

- OS: CachyOS (using LTS kernel, 6.18.42-1-cachyos-lts)
- Bootloader: Limine
- Motherboard: ASUS X870E
- CPU: Ryzen 9 9950X3D
- GPU (primary being used by the operating system): Nvidia RTX 3080
- GPU (attempting to passthrough to the Windows VM): Nvidia RTX 3050

I have largely been following this guide to accomplish this, adjusting the steps as needed for Limine. I will walk through the steps proving that I have followed them to the best of my ability right up until the failure, which is building the virtual machine.

So, from the top:

- Enabling AMDVi in BIOS:
While I can't explicity prove this through code (nor do I know how) I have other VMs running on this computer that should not work if AMDVi is enabled, so I have reason to believe that it's enabled, and I can see in my BIOS that this is the case.

- Kernel parameters:
As stated previously, I am currently using limine. From the /etc/default/limine file, here are my parameters:

ESP_PATH="/boot"
KERNEL_CMDLINE[default]+="amd_iommu=on iommu=1 quiet nowatchdog splash rw rootflags=subvol=/@ root=UUID=561b1a8b-dd95-4558-ba83-773a9c41da2f"
BOOT_ORDER="*, *lts, *fallback, Snapshots"

I understand most places recommend using iommu=pt but this produced an undesirable output in dmesg, namely "Default domain type: Passthrough" which tells me the devices are not being isolated. With the iommu=1 parameter, this message does not appear

- GPU is registered by the system and is in an IOMMU group:

Here is the output of the lspci command the guide recommends. It does seem like both are being picked up and are in separate IOMMU groups:

❯ lspci -nn | grep -E "VGA|Audio" | grep -i nvidia
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA102 [GeForce RTX 3080 12GB] [10de:220a] (rev a1)
01:00.1 Audio device [0403]: NVIDIA Corporation GA102 High Definition Audio Controller [10de:1aef] (rev a1)
0e:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA107 [GeForce RTX 3050 6GB] [10de:2584] (rev a1)
0e:00.1 Audio device [0403]: NVIDIA Corporation GA107 High Definition Audio Controller [10de:2291] (rev a1)

- Binding GPU to VFIO:
My vfio.conf file in /etc/modprobe.d looks like this:

options vfio-pci ids=10de:2584,10de:2291
softdep snd_hda_intel pre: vfio-pci

This appears to match the GPU UUIDs that were displayed as a result of lspci above.

I have also updated the /etc/mkinitpcio.conf file. Here is the output relating to MODULES:

❯ sudo cat /etc/mkinitcpio.conf |grep MODULES
# MODULES
#     MODULES=(usbhid xhci_hcd)
MODULES=(vfio_pci vfio vfio_iommu_type1)

And after a reboot, lspci appears to show that these GPUs are indeed bound to that kernel module

❯ lspci -nnk -s 0e:00
0e:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA107 [GeForce RTX 3050 6GB] [10de:2584] (rev a1)
       Subsystem: ASUSTeK Computer Inc. Device [1043:8976]
       Kernel driver in use: vfio-pci
       Kernel modules: nouveau, nvidia_drm, nvidia
0e:00.1 Audio device [0403]: NVIDIA Corporation GA107 High Definition Audio Controller [10de:2291] (rev a1)
       Subsystem: ASUSTeK Computer Inc. Device [1043:8976]
       Kernel driver in use: vfio-pci
       Kernel modules: snd_hda_intel

From here, the next steps involve creating the VM itself. Here is the XML that I am using to create that VM, keeping in mind that I am trying to pass through the above GPU and using a bridged network connection (i anticipate the need to copy files back and forth through use of a shared folder at some point like i do through my other VM, and turning off networking entirely):

<domain type="kvm">
  <name>win10-passthrough</name>
  <uuid>50c53b52-f386-42aa-bf05-96db518a977b</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://microsoft.com/win/10"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory>16777216</memory>
  <currentMemory>16777216</currentMemory>
  <vcpu>8</vcpu>
  <os>
    <type arch="x86_64" machine="q35">hvm</type>
    <loader readonly="yes" type="pflash">/usr/share/edk2/x64/OVMF_CODE.4m.fd</loader>
    <boot dev="hd"/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state="on"/>
      <vapic state="on"/>
      <spinlocks state="on" retries="8191"/>
      <vpindex state="on"/>
      <runtime state="on"/>
      <synic state="on"/>
      <stimer state="on"/>
      <frequencies state="on"/>
      <tlbflush state="on"/>
      <ipi state="on"/>
      <avic state="on"/>
    </hyperv>
    <vmport state="off"/>
  </features>
  <cpu mode="host-model" check="none"/>
  <clock offset="localtime">
    <timer name="rtc" tickpolicy="catchup"/>
    <timer name="pit" tickpolicy="delay"/>
    <timer name="hpet" present="no"/>
    <timer name="hypervclock" present="yes"/>
  </clock>
  <pm>
    <suspend-to-mem enabled="no"/>
    <suspend-to-disk enabled="no"/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2"/>
      <source file="/home/kobra2112/.local/share/libvirt/images/win10-passthrough.qcow2"/>
      <target dev="sda" bus="sata"/>
    </disk>
    <disk type="file" device="cdrom">
      <driver name="qemu" type="raw"/>
      <source file="/home/kobra2112/Downloads/Win10_22H2_English_x64v1.iso"/>
      <target dev="sdb" bus="sata"/>
      <readonly/>
    </disk>
    <controller type="usb" model="qemu-xhci" ports="15"/>
    <controller type="pci" model="pcie-root"/>
    <controller type="pci" model="pcie-root-port"/>
    <controller type="pci" model="pcie-root-port"/>
    <controller type="pci" model="pcie-root-port"/>
    <controller type="pci" model="pcie-root-port"/>
    <controller type="pci" model="pcie-root-port"/>
    <controller type="pci" model="pcie-root-port"/>
    <controller type="pci" model="pcie-root-port"/>
    <controller type="pci" model="pcie-root-port"/>
    <controller type="pci" model="pcie-root-port"/>
    <controller type="pci" model="pcie-root-port"/>
    <controller type="pci" model="pcie-root-port"/>
    <controller type="pci" model="pcie-root-port"/>
    <controller type="pci" model="pcie-root-port"/>
    <controller type="pci" model="pcie-root-port"/>
    <interface type="bridge">
      <source bridge="br0"/>
      <mac address="56:55:9b:43:42:d5"/>
      <model type="e1000e"/>
    </interface>
    <console type="pty"/>
    <channel type="spicevmc">
      <target type="virtio" name="com.redhat.spice.0"/>
    </channel>
    <input type="tablet" bus="usb"/>
    <tpm model="tpm-crb">
      <backend type="emulator"/>
    </tpm>
    <graphics type="spice" port="-1" tlsPort="-1" autoport="yes">
      <image compression="off"/>
    </graphics>
    <sound model="ich9"/>
    <video>
      <model type="qxl"/>
    </video>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0" bus="14" slot="0" function="0"/>
      </source>
    </hostdev>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0" bus="14" slot="0" function="1"/>
      </source>
    </hostdev>
    <redirdev bus="usb" type="spicevmc"/>
    <redirdev bus="usb" type="spicevmc"/>
  </devices>
</domain>

I believe I'm getting everything right here, including the firmware, however when I click the "Begin Installation" button in QEMU, I get the following error:

Unable to complete install: 'internal error: QEMU unexpectedly closed the monitor (vm='win10-passthrough'): 2026-09-01T16:57:21.699349Z qemu-system-x86_64: -device {"driver":"vfio-pci","host":"0000:0e:00.0","id":"hostdev0","bus":"pci.4","addr":"0x0"}: vfio 0000:0e:00.0: Could not open '/dev/vfio/21': Permission denied'

I believe this is simply because when I look at that group in that directory, it appears to have permission set 600 on it, meaning it can't be read by anything other than root. Since I am creating these in a user session rather than a root session to avoid the possible security risks, that simply means the permissions need to be updated like so:

drwxr-xr-x      - root 31 Aug 22:23  devices
crw-rw-rw-  241,0 root 31 Aug 22:23 󰡯 21
crw-rw-rw- 10,196 root 31 Aug 22:23 󰡯 vfio

Doing that fixes that issue. I know this is not the "correct" way to resolve this, but at the same time I don't know of any other method. However, this then raises another when I click "Begin Installation"

Unable to complete install: 'internal error: QEMU unexpectedly closed the monitor (vm='win10-passthrough'): 2026-09-01T17:00:02.576305Z qemu-system-x86_64: -device {"driver":"vfio-pci","host":"0000:0e:00.0","id":"hostdev0","bus":"pci.4","addr":"0x0"}: vfio 0000:0e:00.0: group 21 is not viable
Please ensure all devices within the iommu_group are bound to their vfio bus driver.'

Now here's where I am kind of at the end of my rope. It appears to me that I have set everything up correctly, but for some reason the VM is not accepting this GPU as a viable one. Can someone please act as my second set of eyes and confirm I am setting everything up correctly? Because at this point, I do not know how to get past this error message.

Any help would be greatly appreciated. I would like to think I didn't sink the funds into this project for nothing.

So iam from ME Embedded Systems in Bits pilani with 7.15 cgpa till second sem and background in electrical engineering and and preparing for digital vlsi side so what can I expect from practice school 2.

Hello, Can some help me with this question?

I have an NVIDIA GPU that I pass through to a Windows 11 VM using QEMU/KVM and libvirt.

I used to have libvirt hooks that stopped the display manager, unloaded the NVIDIA modules, manually detached the GPU with `virsh nodedev-detach`, and then loaded `vfio-pci`.

I recently removed those hooks to see if they were still needed. Surprisingly, passthrough still works fine. When the VM starts, the GPU is used by the VM, and when the VM shuts down, the NVIDIA driver works normally on the host again.

My GPU is configured as a managed PCI device in libvirt (`managed="yes"`).

So my question is: Are these manual NVIDIA driver unload / PCI detach hooks still needed with current QEMU/KVM + libvirt, or does libvirt handle this automatically now?

I'm mainly wondering if there is a reason to keep the old hooks even though everything seems to work without them.

Backstory

I made a few attempts at GPU passtrough for my move away from Windows and had success but it never all worked perfectly smoothly. But this time i finally found a method that does, so i am sharing it in case it works for anyone else.

This guide assumes you already have IOMMU and KVM set up, it is NOT a full guide.

Did it work for you? Please tell us!

A lot of this was figured out with the help of Claude. If you are having trouble copy paste this post into it and explain where you got stuck, it will help you troubleshoot the problem.

This process solves a list of problems: * No longer need to stop your entire Wayland or X11 desktop environment to keep it from touching the GPU during a passtrough transition. * The logs in dmesg being flooded with Nvidia driver initialization attempts when GPU is bound to vfio (NVRM, Nvlink Core, nvidia-nvlink ...etc) * The entire machine freezing up if you attempt to passtrough the GPU when not ready * Crash on re-mounting GPU into linux due to the Nvidia driver setting up the HDMI ports too soon * Monitors on Nvidia GPU showing up on Linux (most people use a HDMI dummy plug, hence it is not a real monitor) * Nvidia GPU burning a lot of idle power when not being used

Tested on hardware: * ASRock X670E PG Lightning (v1.30.AS02) * AMD Ryzen 9 7950X (using iGPU for monitor output) * Nvidia RTX 3090 (using HDMI dummy plug)

OS: ZorinOS 18.1 Core (Linux 7.0.0-30-generic)

Dynamic Nvidia GPU Passthrough (No Desktop Restart Required)

Setup: a Linux host with display running entirely on an integrated/secondary GPU (e.g. AMD iGPU), and a discrete Nvidia GPU that is:

• Passed through to a VM on demand via VFIO, when needed.
• Used on the host the rest of the time for PRIME render-offload gaming — no display output, ever, from this GPU.

The goal: switch the GPU between host and VM without restarting the display manager, without kernel crashes, and without excess idle power draw.

1. Disable Nvidia DRM KMS

By default, nvidia-drm performs full DRM/KMS mode-setting, which causes two problems: the display manager initializes the card as a display device just because it's present (wasting VRAM and blocking clean detach), and re-loading the driver on VM handback can crash the kernel if a dummy/EDID-reporting dongle is plugged into an output (a NULL pointer dereference in nvidia's HDMI/DP audio power path, nv_audio_dynamic_power, triggered via Xorg's DRM hotplug handling).

Disabling KMS via /etc/modprobe.d/*.conf is not reliable — competing config files, install directives, or initramfs staleness commonly override it silently. Use a kernel boot parameter instead, which always wins:

# /etc/default/grub — append to the existing GRUB_CMDLINE_LINUX_DEFAULT line
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash nvidia-drm.modeset=0"

sudo update-grub
sudo update-initramfs -u
sudo reboot

Verify:

cat /sys/module/nvidia_drm/parameters/modeset   # should print N
xrandr --listproviders                          # Nvidia should not appear

This has no effect on PRIME render-offload (__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia %command%), which only needs the render node, not KMS.

2. Enable persistence mode

Without an active display, the Nvidia driver fully tears down and reinitializes between workloads, which is fragile and can cause visible glitches or crashes on repeated switching. nvidia-persistenced keeps the driver state warm instead.

Ubuntu's default unit starts the daemon with --no-persistence-mode, so it must be overridden:

sudo systemctl unmask nvidia-persistenced      # only if masked
sudo systemctl edit nvidia-persistenced

Add:

[Service]
ExecStart=
ExecStart=/usr/bin/nvidia-persistenced --user nvidia-persistenced --verbose

sudo systemctl restart nvidia-persistenced

Verify Persistence-M shows On in nvidia-smi, and confirm it survives a full reboot.

3. Handle idle power / stuck boost clocks

With no display and no active workload, the GPU can get stuck at a high-power state (P0) instead of idling at P8, drawing 100+ W for nothing. A GPU reset clears this reliably:

sudo nvidia-smi -r

This is a known driver quirk, not specific to this setup. Expect idle power to settle around 15–25 W after a reset

4. The switching script

Persistence mode keeps a device handle open, which blocks a clean VFIO detach (NVRM: Attempting to remove device ... with non-zero usage count!). Stop it before detaching, restart it after reattaching, and reset the GPU to clear any stuck boost state:

#!/bin/bash
# gpu-to-vm.sh — hand the GPU to the VM
sudo systemctl stop nvidia-persistenced
sleep 1
sudo virsh nodedev-detach pci_0000_01_00_0
sudo virsh nodedev-detach pci_0000_01_00_1

#!/bin/bash
# gpu-to-host.sh — reclaim the GPU after the VM shuts down
sudo virsh nodedev-reattach pci_0000_01_00_0
sudo virsh nodedev-reattach pci_0000_01_00_1
sudo systemctl start nvidia-persistenced
sudo nvidia-smi -r

No gdm/display-manager restart is needed anywhere in this flow.

Safer script that aborts if the GPU is not completely clear:

#!/bin/bash
# gpu-to-vm.sh — hand the GPU to the VM with abort 
sudo systemctl stop nvidia-persistenced 
sleep 1

if sudo fuser -s /dev/nvidia* 2>/dev/null; then
    echo "ERROR: GPU still in use, aborting handoff (not touching PCI state)" >&2
    sudo fuser -v /dev/nvidia* >&2
    sudo systemctl start nvidia-persistenced   # undo the stop, nothing else changed
    exit 1
fi

echo "GPU confirmed clear, proceeding with detach"
sudo virsh nodedev-detach pci_0000_01_00_0
sudo virsh nodedev-detach pci_0000_01_00_1

5. Optional cleanup

If the GPU has a dummy HDMI/DP dongle for the VM's benefit, the host will still enumerate a dead audio function for it. To silence it (cosmetic only, not required for stability):

# /etc/udev/rules.d/99-nvidia-hdmi-audio-noprobe.rules
ACTION=="add", SUBSYSTEM=="pci", KERNEL=="0000:01:00.1", ATTR{driver_override}="none"

Diagnostics reference

nvidia-smi                                   # power state, clocks, processes
sudo fuser -v /dev/nvidia*                   # what's holding the GPU open
cat /sys/module/nvidia_drm/parameters/modeset # confirm KMS is off (N)
xrandr --listproviders                       # confirm Xorg isn't using the GPU

Useful quick test for PRIME offloaded GPU rendering (shows a desktop window with 3D gears rendered by the Nvidia GPU):

DISPLAY=:0 __NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia glxgears

Hi, I saw that vGPU can be used on RTX 3000 series cards; is it possible to allocate VRAM to a Windows VM and get good performance ?

Intro

I have been trying to make the switch from Windows to Linux for a while. I got GPU passtrough mostly working in the past but it was never really smooth. However now i seam to have got it.

This is not a full tutorial, but might serve as useful hints for people that are trying to do this on similar hardware.

Hardware: * ASRock X670E PG Lightning (v1.30.AS02) * AMD Ryzen 9 7950X (using iGPU for monitor output) * Nvidia RTX 3090 (using HDMI dummy plug) * Samsung PM9A1 1TB (Windows boot drive)

Setting Up GPU Passthrough on ZorinOS

This covers getting from a fresh ZorinOS install to a working Windows VM with a discrete Nvidia GPU passed through on demand, while still being able to use that GPU for host-side gaming (via PRIME render offload) the rest of the time.

For the follow-up problem of switching the GPU back and forth without restarting the desktop, fixing a KMS-related kernel crash, and handling idle power draw, see the companion guide: https://www.reddit.com/r/VFIO/comments/1w2otjo/my_live_gpu_mountunmount_script_on_zorinos_fix/

1. BIOS setup

Enable IOMMU (AMD-Vi) in the BIOS. On an ASRock X670E PG Lightning specifically: Advanced > AMD CBS > NBIO Common Options > PCIe Access Control Service This option can be hidden until "PCIe Advanced Error Reporting" is enabled first.

2. Enable IOMMU ACS override on the kernel command line

Editing the grub config: bash sudo nano /etc/default/grub GRUB_CMDLINE_LINUX_DEFAULT="quiet splash" bash sudo update-grub sudo update-initramfs -u sudo reboot Only add pcie_acs_override=downstream,multifunction if the next step shows your GPU sharing an IOMMU group with unrelated devices — it weakens device isolation, so skip it if you don't need it.

3. Identify the GPU and confirm IOMMU grouping

bash lspci -knn | grep -A3 NVIDIA Note the PCI address (e.g. 01:00.0 for the GPU, 01:00.1 for its HDMI audio function) and vendor:device IDs (e.g. 10de:2204, 10de:1aef).

bash find /sys/kernel/iommu_groups/ -type l Confirm the GPU and its audio function land in their own IOMMU group, ideally with no unrelated devices alongside them.

On the ASRock X670E PG Lightning a lot of the M.2 slots DO NOT have correct IOMMU groupings (the one directly connected to the CPU does) so the Windows boot drive must be in that one.

4. Load VFIO kernel modules

bash echo "vfio vfio_iommu_type1 vfio_pci vfio_virqfd" | sudo tee -a /etc/modules

Note: don't permanently blacklist nvidia/nouveau or statically bind the GPU's IDs to vfio-pci in /etc/modprobe.d/ unless the GPU is dedicated to the VM full-time. For a setup where the GPU is also used on the host, let the nvidia driver load normally at boot, and hand the device to VFIO dynamically per-session (see companion guide) — this is the approach that ended up working reliably here; an earlier attempt using permanent blacklisting plus manual modprobe -r / display-manager restarts also works but is clunkier and unnecessary for dual-use setups.

5. ZorinOS Conveniance setup

• sudo apt install openssh-server — useful for remote access once the display state gets manipulated.
• Install x11vnc and run it as a systemd service for remote desktop access. x11vnc requires an X11 session, so:
• Switch the desktop session from Wayland to Xorg: log out, click the gear icon on the login screen, select the Xorg session.

6. Install the virtualization stack

bash sudo apt install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils virt-manager ovmf

7. Pass through a physical NVMe drive and install Windows

bash sudo virsh nodedev-detach pci_0000_18_00_0 # detach the NVMe from the host In Virtual Machine Manager: create a new VM, uncheck "Enable storage" (to use the raw NVMe device instead of a virtual disk), check "Customize configuration before install", set Firmware to OVMF_CODE_4M.fd (the AMD SEV firmware variant had problems booting from NVMe), and add the NVMe drive under Add Hardware → PCI Host Device.

Install Windows 10 onto the physical drive through the VM.

Afterwards, reboot the host directly into that Windows install (outside the VM, GPU still available natively) and install the latest Nvidia drivers — this avoids driver-installation quirks that show up when installing while running inside a passthrough VM for the first time.

bash sudo virsh nodedev-reattach pci_0000_18_00_0 # give the NVMe back to the host sudo update-grub # should detect the Windows install as a dual-boot entry

8. Add the GPU to the VM

In the VM's hardware settings: Add Hardware → PCI Host Device → add both the GPU and its audio function.

Edit the VM's XML (virsh edit <vm-name>) and: - Set managed='no' on the GPU's <hostdev> entry, so libvirt does not automatically bind/unbind the device at VM start/stop — this setup manages that manually via nodedev-detach/nodedev-reattach instead, which is what makes host/VM switching possible without recreating the VM. - Add inside the <features><hyperv> block: xml <vendor_id state='on' value='1234567890ab'/> This hides the hypervisor from Nvidia's driver and avoids the Code 43 error in the guest.

Under Boot Options, set the NVMe disk as the boot device.

9. Dynamic switching, crash fix, and power management

With the above in place, the VM boots and the GPU works inside it. The remaining problems — avoiding a gdm restart on every switch, a kernel crash on GPU reattach caused by a dummy HDMI dongle plus KMS re-initialization, stuck high-power idle states, and a clean detach/reattach script — are covered in the companion guide: nvidia-dynamic-vfio-passthrough-howto.md.

Example system (used throughout this guide and its companion)

Substitute your own PCI addresses and IDs throughout — get them from lspci -knn and find /sys/kernel/iommu_groups/ -type l on your own system.

Hi, I built my own VFIO passthrough script.

It sets up GPU passthrough and works around the usual AMD problems: wrong device bind, dead card after VM stop, and a guest that cannot reboot cleanly. On my RX 9070 the VM can start, stop, and reboot without me rebuilding the host by hand every time.

VFIO helper for AMD RX 9070 / RDNA4 — live-attach, reboot, and host-safe bind

Hi, I wrote a VFIO passthrough helper because the usual guides were not enough for an RX 9070.

The script is a host setup wizard, not a VM manager. It can:

bind only the guest GPU by PCI address

use dynamic bind / live-attach so the card is not killed at VM start

keep the host desktop on the other GPU

toggle hotplug on/off without replacing the whole VM XML

optionally set up Looking Glass, virtio-win, and VM tuning

It does not officially fix AMD’s reset bug. It works around it so the card stays usable.

Hardware I tested: dual GPU, RX 9070 as the guest card, Linux host (Fedora / openSUSE).

Motherboard: Gigabyte Z790 UD, vF15

GPU: AMD Radeon RX 9070 XT

CPU: Intel i7 12700K, UHD Graphics 770

Fedora Linux 44 KDE

Kernel: 7.1.10-200

EDIT: Hell, even UHD Graphics 770 for the Virtual Machine will do, I don't really need the RX 9070 XT. Whatever's easier and safer.

Hello, I'm using VMware Workstation Pro 26H1

I noticed with my latest project which I use a lot of versioning for, "input lag" becomes more significant over time.

The more I snapshot, the longer the response time is when keystrokes are being typed with the keyboard.

I have no clue why but I'm pretty sure there are technical reasons why this happens.

All of my snapshots are "directly linked chronologically", there's no tree if that makes sense. They go in a straight line.

https://i.imgur.com/QHDk8yN.png

It becomes very obvious after 40 or so snapshots, I would estimate the lag to be almost 1 full second by that time, and it's very unsustainable.

Is there a setting I should check in the options, or in the config files, to reduce or getting rid of this problem?

Thanks

Short version: you can give a pre-UEFI guest a working passed-through GPU without ever touching legacy VGA routing, which means it works in any slot and several such guests can run simultaneously. I have two Windows XP SP3 seats and two Windows 7 x64 seats running at the same time on one host, each with its own card, monitor and USB controller.

Why the obvious approach fails

XP predates UEFI, so it wants legacy BIOS services. But a legacy BIOS initialises a graphics card by executing that card's legacy VBIOS, and that VBIOS needs working access to its own card's legacy VGA registers. vfio-pci's VGA region does real inb/outb on the host after taking the arbiter lock, so the host chipset has to physically route those cycles to the target card — and on a multi-slot board that generally only happens for the one card the host firmware POSTed from. Any other card gives the guest Code 10.

I measured this directly. With x-vga on a second card, bridge VGA Enable (bit 3 of BRIDGE_CONTROL, offset 0x3e) flips to the second card's upstream bridge and the arbiter reports owns=none — a second x-vga guest steals routing from the first. So even where legacy works, it's inherently a single-seat resource. Also worth knowing: the NVIDIA legacy VBIOS hangs not just on a cold card but whenever a foreign VGA core answers the legacy ports — it hangs identically on the warm, known-good card if you let an emulated adapter answer.

(Side note for anyone reading /dev/vga_arbiter: use head -1, never cat. cat blocks forever.)

What works instead

Boot OVMF, so the card is initialised by its own UEFI GOP driver out of its option ROM — no legacy VBIOS execution, no VGA arbitration, no x-vga. Then chainload CSMWrap (SeaBIOS packaged as a UEFI application), which supplies the legacy BIOS services the old guest needs. The guest boots and finds a card that is already initialised.

Consequences, and this is the useful part:

• works in any slot, including cards on a second root complex
• no x-vga anywhere, so there is no single-owner resource to fight over and multiple seats run concurrently
• vga: std (an emulated adapter) must stay for OVMF's own console. Once the guest's driver takes the real card, the emulated console goes black — that's the success signal, not a fault
• Windows 7 needs a different shim, not CSMWrap: UefiSeven, which fabricates a fake Int10h/VESA handler for bootvid and then chainloads the real boot manager

Guest timing: the trap that isn't an error message

If you get a pre-UEFI Windows guest booting but its ACPI is unhappy, the standard advice is to switch it to a non-ACPI HAL. Don't. Without ACPI, XP has no PM timer and falls back to the PIT, which no hypervisor ticks precisely enough for game engines — everything runs in slow motion, silently. kvm-pit.lost_tick_policy=discard and -rtc driftfix=slew reduce it and do not fix it. It also pegs a host core per guest vCPU and breaks clean shutdown. On the ACPI HAL the same guest idles at 0.6% of a vCPU with halt_exits climbing normally.

ROMs: one per physical card, passed whole

• Dump each card's own flash. Two identical Quadro K4000s here carry different firmware revisions (80.06.76.00.0B vs 80.06.38.00.03), and using the sibling's ROM produced visible framebuffer corruption. The ROM follows the card, not the slot.
• The dump must contain both an x86 legacy image and a UEFI/GOP image. Strip to GOP-only and OVMF refuses to drive the card; strip to legacy-only (or rombar=0) and it's never POSTed → Code 43. Verified both ways.
• A boot_vga=1 card's sysfs ROM is a truncated legacy shadow copy with no UEFI image — it looks fine and fails hours later. Read the real flash through the card's PROM window instead (for NVIDIA: mmap BAR0, clear 0x1850, read 512 KB at 0x300000, restore; 0x1850 reads back 0xbad0011f, which is normal).
• Set driver_override before unbinding or nouveau grabs the card instantly, and remember that reading rom while bound to vfio-pci silently returns 0 bytes.

There's a tool in the repo that does all of that and validates the result:

sh ./setup/gpu-rom-dump.py --list ./setup/gpu-rom-dump.py 0000:0a:00.0 -o mygpu.rom --install ./setup/gpu-rom-dump.py --check /usr/share/kvm/mygpu.rom

CSMWrap-specific gotchas, whatever you drive QEMU with

• Its SeaBIOS crashes the VM (KVM internal error. Suberror: 1) if the CSM boot device is USB-on-xHCI, AHCI/SATA or NVMe. The BIOS-proxy helper AP enters its first dispatched call with ESP = 0, pops the real-mode IVT as a return address and jumps into unbacked MMIO (EIP=f000ff53). It is racy, not deterministic — one config passed once and crashed the next run. PIIX IDE on i440fx is the only well-evidenced safe bus; virtio-blk also survived. Register dumps, a bus matrix and a bare-QEMU reproducer are in the repo, with a report drafted for upstream.
• CSMWrap ≥ 2.0 permanently reserves one logical CPU as that helper and spins it on a mailbox loop that never halts, hiding it from the guest's MADT/MP table. Budget one pegged host core per seat, and never give it 1 vCPU — 3.x panics outright.
• Use 3.1.2 or newer. 1.3.0 avoids the helper crash but can't boot XP.

Input latency, since it's a passthrough question

Mapping individual USB devices into a guest routes every click through the host's translation layer, which is fine for a desktop and miserable for anything twitchy. I used a StarTech PEXUSB3S44V — four independent USB 3.0 host controllers on one PCIe card, each in its own IOMMU group — and passed one whole controller per seat. Bare-metal input, and one seat spamming or crashing can't disturb another.

The one Proxmox-specific wart

Proxmox adds two pci-bridge devices to every i440fx VM unconditionally, with no config option to suppress them, and XP's acpi.sys bugchecks 0xA5 (0x00000002) (ACPI_ROOT_PCI_RESOURCE_FAILURE) on them under CSMWrap. Proven by running PVE's own generated argv by hand with only those two -device lines removed: XP boots to the desktop. Occupying them with pci-testdev, giving them prefetchable BARs via virtio-rng-pci, and older machine types were all tested — it's their presence, not their windows.

If you drive QEMU or libvirt yourself, this is a non-problem — just don't add them. On Proxmox I ended up diverting /usr/bin/kvm to a wrapper that strips those two argument pairs for allowlisted VMIDs; details in the repo if you're on PVE.

Links

Repo: https://gitlab.com/zubrjan/2am-idea — both recipes as step-by-step guides, the ROM tool, the wrapper, all four VMs' real configs, and the full debugging record. MIT.

Video: https://www.youtube.com/watch?v=cfP1x4R3XEc — XP seats on 3DMark06, Win7 seats on Unigine Heaven. Three monitors on camera because that's how many I own; the fourth seat is switched onto one of them partway through, with the host view on a laptop.

All verified on exactly one host: dual Xeon E5-2667 v2, 2× Quadro K4000 (XP), Quadro P2200 + GTX 1060 6GB (Win7), QEMU 11.0.0, OVMF/edk2 4.2025.05, CSMWrap 3.1.2, UefiSeven 1.30. That's the honest limit of it. If you try this — especially with AMD cards, or a different pre-UEFI guest — I'd like to hear what happened.

f you've tried running Sunshine inside a Hyper-V VM with a GPU partition (Add-VMGpuPartitionAdapter), you've probably hit this:

IDXGIOutput1::DuplicateOutput()  ->  0x80070057

The paravirtual display adapter doesn't implement Desktop Duplication. Changing topology, monitor count or primary display doesn't help — it's architectural.

But Windows.Graphics.Capture works fine, because it goes through DWM instead of the display driver. Same output, same moment:

DuplicateOutput()       ->  0x80070057
CreateForMonitor()      ->  S_OK, frames flowing

Sunshine already ships a complete WGC backend — it just filters every output through a Desktop Duplication probe first, so WGC never gets a chance. Removing that gate when capture = wgc is set explicitly is the whole fix.

Result (RTX 4090 shared to a Win10 LTSC guest):

5120x1440 @ 60, HEVC, hardware NVENC
~10 ms latency, ~1.5 ms client decode, ~5% host CPU
Vulkan 1.4.325, OpenGL 4.6, Direct3D 12, full VRAM

Two things that cost me the most time and aren't obvious:

• Sunshine can't run as a service. As SYSTEM it can't activate WGC — IsSupported() throws. It has to run as the logged-on user, which makes auto-logon mandatory.
• The Hyper-V display has to stay active. Deactivate it and native Vulkan and OpenGL die 13–32 s later. Direct3D keeps working, so it's a misleading indicator. I park the Hyper-V display diagonally below-right so the desktop origin stays at (0,0).

I packaged the whole setup as a one-click installer — patched Sunshine, virtual display driver, scheduled tasks, and UIAccess so UAC prompts are actually clickable over the stream:

https://github.com/VjekoslavDugec/luxstream-setup

Known gaps: gamepad is untested with a real controller, and I haven't measured concurrent NVENC session limits across cloned VMs.

Patch is a small diff against upstream Sunshine — happy to open a PR if there's interest.

recently got back into doing VMs, however after I install Windows to the NVME drive that I added as a "PCI Host Device" then when I reboot my pc the UEFI of the host motherboard detects the EFI partion and adds it to the boot order as primary boot device, I can manually remove it from the boot order, but it might get set back as the primary boot device in the future, which is annoying

also efibootmgr lists the drive as an efi variable, I could delete it but the motherboard will probably add it back,

also the motherboard I'm using is an ASUS Prime B550 Plus, the firmware version is 3636

is there another way of adding the nvme drive that causes the motherboard to NOT detect it or what? is it even possible to change this, it's rather annoying

also I have no plans to use this drive for baremetal booting

I am running qemu 11.1 and virt-manager on a system with RX 470 and i5-10500 with UHD 630, board is Gigabyte B460M D3H. RX 470 is used for host's output. The guest is Windows 10 19041.1 and has Intel graphics driver 31.0.101.2141. my cmdline has intel_iommu=on iommu=pt, /etc/modprobe.d/vfio.conf is 'options vfio-pci ids=8086:9bc8', /etc/modprobe.d/blacklist.conf is 'blacklist i915'. /etc/modules-load.d/vfio.conf is 'vfio-pci'. EDIT: I tried adding video=efifb:off too to my host cmdline.

vfio-pci shows only:

[20699.060075] vfio-pci 0000:00:02.0: resetting
[20699.163736] vfio-pci 0000:00:02.0: reset done

in dmesg, no errors.

My virt-manager xml is as follows: https://pastebin.com/tPNq7huW

I have tried adding the GPU's ROM and enabling x-igd-opregion and x-igd-lpc. I am using CFL_CML_GOPv9.1_igd.rom.

Tested linux, doesn't work either

I am unable to get Rust to work on my vfio setup. A while ago, copying smbios from my motherboard was enough to get it to work. Now, EAC won't launch. Can someone take a look at my XML and tell me if I'm missing anything obvious, or if this is still possible in the first place? My host machine is Gentoo, with QEMU 11.1.0, libvirt 12.0.0, and virt-manager 5.1.0. TYIA

The app keeps running on Windows. There's a virtual display on each machine — the window gets moved onto the Windows one, captured, and drawn on the Mac as a real macOS window next to my Mac apps.

Median just under 10 ms, capture to display, wired.

NVIDIA only for now. Three months, solo.

I used to be able to do GPU passthrough - my problem ever since upgrading to the latest distro seems to be because of simple frame buffer where as it used to be vesa. I’ve tried multiple solutions such as blacklisting the simple frame buffer on my .conf files
It doesn’t seem to work. I’ve also changed my grub files to make sure it is using my desired frame buffer however when I login into my wayland session - it still shows as simple frame buffer.

Anyone have any idea on how to get around this? Or maybe an updated script for start stop for my 4070 super to do a gpu passthrough using simple frame buffer? Thank you.

After two weeks of trial and error, it looks like I am very close to successfully establishing a proxmox node with the following passthrough scheme:

• [iGPU]: AMD 7950X3D iGPU → Linux VM
  • Development + general use
• [dGPU]: RTX 4090 → Windows VM
  • Docker under WSL2 for all home services (home assistant, ollama, jellyfin, etc)
  • Occasional gaming

I have run head first into several brick walls and needed to clarify for myself basic aspects of how PCIe virtualization works.

I'm having one more bizarre issue that I might need help with. Suppose the following configuration

• iGPU
  • included in vfio-pci.ids
  • amdgpu is blacklisted in modprobe.conf
• dGPU
  • excluded from vfio-pci.ids
  • no blacklist in modprobe.conf

This boots successfully. I can see the tty on the dGPU outputs, and I also have ssh access. Then, I try:

• dGPU
  • included in vfio-pci.ids
  • nouveau, nvidia, and nvidia* are blacklisted in modprobe.conf

No logs from initramfs or early boot are visible on the dGPU outputs (expected). The displays are completely undriven for both (expected). The device does not connect to the network, so I can't ping it. (unexpected)

I know that the dGPU and its associated audio device share one isolated IOMMU group. I do not expect that its inclusion in vfio-pci.ids should have any effect on the system. If the device is driven by vfio-pci, then I also expect the modprobe blacklist would be benign as well.

The only idea I've arrived at is that booting without a GPU would disrupt Proxmox's boot at least enough to prevent the network system to come up? To my understanding, it should be possible to run a Linux kernel in a completely headless context, so I don't know why this one change would result in things breaking.

Any guidance or suggestions would be appreciated.

1. Other kernel cmdline args:

• disable_vga=1
• initcall_blacklist=sysfb_init
• iommu=pt
• video=efifb:off
• vfio-pci.ids which are included in both cases
  • All USB controllers and network controllers, except one USB controller which has its own isolated IOMMU group (USB C -> Ethernet for proxmox host)
  • All thunderbolt, SATA controllers
  • Some kind of encryption accelerator PCI device, not sure what could make good use of this

1. Softdep (modprobe.conf) on vfio-pci exists for all modules

I'm trying to move all my gaming to Linux. Initially I asked for help at r/linux_gaming, but some people saw negatively that my setup was not a bare-metal gaming distro and some others recommended I posted here. You can check my post there if you want to read any of my comments. But I'm trying to gather all the useful info here.

My host is:

• OS: Unraid
• CPU: AMD Ryzen 9 3900X
• RAM: 128 GiB DDR4

With KVM + QEMU + Libvirt I'm running a VM:

• Operating System: NixOS 26.05
• KDE Plasma Version: 6.6.6
• KDE Frameworks Version: 6.26.0
• Qt Version: 6.11.1
• Kernel Version: 7.1.4 (64-bit)
• Graphics Platform: Wayland
• Processors: 6 cores, 2 threads of AMD Ryzen 9 3900X
• Memory: 48 GiB of RAM (hugepages of 1G)
• Graphics Processor: NVIDIA GeForce RTX 4070 (driver 595.71.05)
• Manufacturer: QEMU
• Product Name: Standard PC (Q35 + ICH9, 2009)
• System Version: pc-q35-10.2

The VM has three storages:

• Primary drive. A qcow2 file in the NVME drive of the host.
• "Fast" Virtiofs device mounted. It's a share of the host that uses the NVME drive.
• "Slow" Virtiofs device mounted. It's a share of the host that uses the NVME drive as cache and the array of HDD drives as main storage.

I prefer to install games in the Virtiofs devices to be able to have that in common with other VMs. But I've tried the primary drive for the problematic games too.

The games I'm trying to run with good performance and failing at that are The Binding of Isaac and Elden Ring: Nightreign. Both with mods. In Isaac I use a bunch of quality of life mods. Nothing that adds characters, floors, bosses, enemies... And not using Repentogon. For Nightreign I use the More Map Variations & Weapons Mod.

The Binding of Isaac takes 15-20 minutes just to start. And then, during gameplay, there are slowdowns and micro freezes anytime something new happens or I change rooms. I guess it's shader compilation, but if it's that, the cache of shaders is not working because it happens again any time the game starts again. And uninstalling the mods doesn't improve things.

Elden Ring: Nightreign has just low FPS. 15-20 fps is the average. But my CPU rarely goes above 60% with no single core going above 85% and my GPU doesn't go above 50% either. The game doesn't work online without the mod because of Easy Anti-Cheat and VMs. So I cannot test it properly without it.

Other games I've tested:

• Control Ultimate Edition: +70 FPS
• Dark Souls III: 60 FPS
• Heroes of the Storm: ~60 FPS
• Lies of P: ~75 FPS
• Outer Wilds: +100 FPS
• Satisfactory: ~50 FPS

HVM and IOMMU enabled. Cores of the VM are isolated in the host. CPU and GPU are in performance modes during gaming. I've run some benchmarks too. I'll put them in a comment so that this is not even longer.

Any idea of what I can do to improve the performance in what are probably 2 of my 3 most played games? (I was really unlucky here I guess)

Title pretty much.
I know that anti-cheat implementation comes down to devs, to block VMs or not (EAC simply detects it, and blocks it by default, but this can change in settings)
Wardogs devs announced they will support Proton and Linux, so I was just would like to try it till then.

Hi everyone, my host computer is Windows 11 and has a Ryzen 7 5700X3D + GTX 3070 + 32GB of RAM.

My virtual machine is running Windows 10 Pro (8GB VRAM allocated), and I can't get rid of the screen tearing inside it.

In the host machine, I already added vmware.exe & vmware-vmx.exe to the NVidia Control Panel and set "Vsync > On" + "Triple buffering" and "fixed frame rate" instead of "G-Sync" but no can do.

VMTools is installed already.

Would someone have a solution for that please?

Thanks very much

Edit: Something strange I noticed is: By disabling "3D acceleration" in VM settings, there is no screen tearing anymore, but then the performance are very suboptimal.

urbanyl/GPU-Passthrough-for-KVM-QEMU-on-Linux: Complete GPU passthrough setup for KVM/QEMU on Linux, enabling near-native Windows VM performance with dedicated GPU access. Covers NVIDIA and AMD GPUs with comprehensive troubleshooting.

Really want to use the training packs in there but don’t have a gaming PC. Am I able to run rocket league somehow this way via a streaming service like shadow pc and run mods with it?