QEMU 10.1.0

KVM is basically three components.

* An abstraction over second level page tables to map some of a host user process as what the guest thinks of as physical memory.

* An abstraction to jump into the context that uses those page tables, and traps back out in the case of anything that the hardware would normally handle, but the hypervisor wants to handle manually instead.

* A collection of mechanisms to handle some of those traps in kernel space to avoid having to context switch back out to the host user process if the kind of trap is common enough, both in the sense of the trap itself happens often enough to show up on perf graphs, as well as the abstraction being exercised is relatively standard (think interrupt controllers and timers).

Let me know if you have any other questions.

Where could someone get started in terms of reading material to learn more about this in depth?

I would assume sooner or later you're going to end up in the Intel Developer manuals or the equivalent for whatever architecture you are interested in. The Intel ones are very complete at least.

> I would assume sooner or later you're going to end up in the Intel Developer manuals or the equivalent for whatever architecture you are interested in. The Intel ones are very complete at least.

I can vouch for this. I'm no virtualization expert but I did stumble upon some intel developers manuals (truthfully, i fell into the rabbit hole) and just skimming it made everything make much more sense.

For example: https://www.intel.com/content/dam/www/public/us/en/documents... - "CHAPTER 23 INTRODUCTION TO VIRTUAL MACHINE EXTENSIONS"

The link above explains how the VMX extension work on intel processors. Any software doing hardware-assisted virtualization (so no binary translation, no full-system-emulation) will likely be using those instructions.

From a different direction, I'd suggest https://www.devever.net/~hl/kvm

I thought part of vt-d/vt-x made the "virtual tables" actual tables.

Eg- the memory the VM can access is controlled by the MMU of the CPU (below ring0/kernel). Resulting in the only VM escapes being the Shim(s) for talking with the host (network, memory balloon, graphics).

The AMD Processor Programming Reference manuals are also good for this, if you like complete and detailed. They complement the Intel manuals. Much the material is duplicate because the processors are so similar, but written in a different way.

How does nested KVM work? Are all the page tables handled by the top level? Do the traps have to propagate up?

if you want to look at existing implementations on top of kvm then these might be useful - rust-vmm is a core library for AWS' firecracker vmm.

https://github.com/rust-vmm/kvm https://github.com/kvmtool/kvmtool https://github.com/sysprog21/kvm-host

Yes, there are virtualization-specific page tables that convert guest physical to host physical addresses. KVM still haw to take host userspace's virtual addresses, convert them to host physical addresses, and make sure that the virtualization-specific page tables stay in sync with the kernel's usual page tables (which convery host virtual addresses to host physical)

Yes, the top level uses write protection of guest memory to combine the two levels of translation into one.