What Is Qubes OS?

Anyone use this as a daily driver? I tried installing it and it crashed on first run. Should have looked at the list of compatible laptop models first. It’s a bit overkill for my needs. My threat model doesn’t require me to spawn a disposable Fedora VM just to read a PDF document. I just open a PDF in Google Docs.

I tried probably half a year ago, and it installed fine, but I just couldn't wrap my head around how to use it right.

Used extensively by Mullvad VPN for a lot of their infrastructure

https://mullvad.net/en/blog/2022/6/15/mullvad-is-now-continu...

Daily driving for years now. Only thing to really keep in mind is having sufficient RAM. Otherwise, it's great for development. You can keep TemplateVMs for all of your development environments and tear them up and down, duplicate them, assign to a VPN, etc. Not good if you need GPU acceleration for anything, but some people have worked on GPU passthrough.

I have in the past before I became bound to doing windows-compatible development. It was actually really great. I didn't hate it at all.

I liked the ability to run multiple linux distros and a windows 7 VM for stuff that needed that, but scrubbing PDFs I think is one of those underrated things considering how much malware comes in through those. Like I would rather not do that in a docker container of all broken condoms. Right now I just have a seperate computer to take care of that. I'd probably use qubes if I had an intel laptop as my daily driver again.

Oh and the only other thing was laptop battery life. Maybe an hour and a half tops.

Yeah 16gigs+ is what you want here. Not rare in modern computers.

same here. the entry bar is really high on qubes.

I remember seeing that Edward Snowden uses it daily. Wonder if that's still the case.

I have for about five years. Install has been fine for me across three laptops (various ThinkPads), with the caveat that I chose models known to work well with linux (you’re booting into fedora, which runs Xen as dom0). Also, the one time I had to do a lot of work was when I bought a newly released version of a laptop; a few months later I upgraded to a later version of Qubes and it installed normally.

There is an up front investment in figuring out how to partition your computer use/apps into VMs and then setting up the VMs. If you’re not already a Linux user there is also the usual learning curve of switching to Linux (most qubes users use mostly Linux vms, windows takes more work to get going, I have windows 10 working but it took some effort).

I absolutely love the disposable VM model. I do all my web surfing (except some financial sites) in disposable VMs and cannot fathom going back to downloading and executing untrusted code (JavaScript) outside a dispVM. Similarly, I cannot imagine opening documents from untrusted third parties outside a vm of some sort. Even software I don’t fully trust (e.g. Zoom, bluRay ripping software) I like to run in disposable VMs or at least their own dedicated vm.

Qubes is like any other specialized tool - it’s worth investing the time if what it offers (security and privacy) is something you especially value. Having seen supposedly exotic and advanced threats become more commonplace over the last 20 years I think we all will end up using systems to some extent similar to Qubes, at least inspired by Qubes. Some of what’s not in your threat model today will be, eventually. The only question is how much.

In practical terms, it is in some ways like going from having one computer to having a network of computers. You do become something of a sysadmin. There is some pain there especially up front but I am at the point where I am expert enough that the ongoing time and pain investment is quite minimal.

More than anything, I feel completely exposed on other OSes. I wish other operating systems (like macOS) would steal the best ideas from qubes. For example, let people open files in disposable VMs when they want to, and cause this to happen by default for downloaded files, and by default have people surf the web in the rough, more seamless equivalent of a disposable VM, possibly with some carve outs for ease of use (like make it almost transparent, with some red flag, to move downloads out of the browser vm, and do likewise with uploads). Also, Qubes has “vaults,” which are just VMs with no internet where you put your most sensitive files; I put basically all my files there because they really don’t need live internet. You could translate this on a “regular” OS into some kind of area that’s extra protected from other processes somehow. For example unprompted access to files in the vault would require explicit authorization, and files in the vault could not cause network connections by default. Something along those lines.

I have been using it for over 5 years for all personal things like email, banking, and paying bills. Once you find good hardware for the OS, it runs very well, but you either need a lot of memory or to close each VM as soon as you're done with it and run only one-two VMs at a time. I would say minimum of 16 GB RAM with 32-64 GB preferred.

Works fine on an older ex-windows laptop, repurposed for throwaway VMs, trying things... Could not get it to run on a 2015 MacBook Pro, would be using it more if I had.

QubesOS is my favourite technology existing today.

Daily driver on desktop and laptop.

Feels like home.

^ My highest praise.

Not really for infrastructure though? Still neat.

I couldn't agree more. Secure computing adoption requires easy usability.

We helped push technical adoption through skeuomorphic design patterns, but left engineers to figure out how to educate users on permissibility. That's a failure on us as an industry. We should be building to keep people safe from the dangers we all know about FIRST, then and only then should we build the access controls to allow access to other resources and interoperability.

I feel like chromiumos is the closest we have to a mainstream solution for this, but a combination of Nix and Qubes would be even better.

Writing from Qubes right now. x230 with 16gb ram and it runs just fine. Still figuring some things out tho.

My main problem with QubesOS is GPU acceleration. Using any intensive app is a chore because its so slow, and I Also game on Linux.

But In general I don't think its for me anyway, I'm comfortable with my current Fedora 36 Workstation setup.

Conceptually, I love it. I used it since about 2016 until last year, but I had to record some video and use stuff like OBS and it just became impossible (with my skill level) to get working.

I abandoned and went back to Fedora, which is odd as I’d stuck with it through lots of other NVIDIA crap issues and such.

Hopefully adoption increases and one day I can use in a workplace setting.

Maybe this isn't the best place to ask this, but I'll try anyway:

I'm a consultant involved in cybersecurity who often has to build and run VMs to either test out software, run things in sandbox, or connect to TOR from a VM I'll never use again.

Having said that, I currently use Windows with VMWare Workstation, but I find it frustrating and would prefer something that's less frustrating and feels more built-in.

Is there a solution that anyone would recommend for this kind of thing? Internal networks, Windows and Linux sandboxes, etc. I use Microsoft office products regularly, and my workstation (Dell Inspiron with an i9, 64GB ram, 2tb SSD) is connected to a thunderbolt 4 dock with 2 1440 monitors. I'd prefer for a Windows VM to have passthrough to the monitors and be able to interact with the host OS via that VM, so I can still share my screen during meetings and while coordinating efforts.

I don't known of this works with all your criteria, but you might want to go with UnRaid or Proxmox or a Type 1 hypervisor like vSphere/ESXi or Xen.

You don’t really mention specifically what you find “frustrating” about VMWare Workstation so it’s hard to know on what criteria to give a response.

I don’t know how “built in” it can be considered but I’ve used LXD a bit and since it now supports VMs as well I’m guessing you could define VMs in yaml in advance and “easily” (depending on your definition) tear down and re-deploy VMs with preconfigured network settings etc. Vagrant should also work for this with a Virtualbox or VMware backend (paid feature).

What exactly do you mean when you say that the VM should be able to “interact with the host OS”, isn’t that exactly what you don’t want and why you’re running a VM in the first place?

I'd like the ability to drop files to a VM from another VM, like shared folders in Workstation.

My frustrations with VMWare usually revolve around network connectivity issues. My internal or NAT networks often fail to give the guest VMs the expected connectivity.

Maybe Fedora with Xen is the route I should try, assuming I can give the Windows VM full GPU pass-through and use it as a "primary" machine. I need to be able to screenshare almost daily via Zoom.

I don't get the distinction between type 1 and type 2.

E.g. xen is type 1 and KVM is type 2. But at the end of the day it's a Linux kernel in both cases that runs the virtual machines, so what's the point of distinction?

I was reading about Device Isolation but there's still something I'm not clear on:

Does the OS claim to prevent partially-trusted PCI devices linked to one VM from accessing memory of another VM? If so, how's that done?

I understand by default the hypervisor resets a device when it's moved from one VM to another, which would mitigate an evil device driver in the former from impacting the latter. But that doesn't protect from isolation breaches caused by evil [persistent] firmware.

I thought PCI cards have DMA access to all the system's memory space, unless you happen to have a server-type motherboard with a "smart PCIe bridge that can be programmed to perform address translation and access restrictions" (https://superuser.com/a/988179). Is such hardware more common now? Or does Qubes rely on all hardware you plug into it being trustworthy?

The iommu device is present on nearly all systems these days, even consumer ones. Intel calls it vt-d. The big issue is the device groupings that are setup by the firmware, and down stream pcie bridges. It's become more common because it's the only way to secure thunderbolt ports

It's what runs above the vms that is the distinction. For xen it has its own kernel instead of running Linux as the hypervisor and host system. Xen still uses Linux typically as the domain zero as it calls it for doing control and setup but it doesn't necessarily have full access to all the hardware on its own.

I do. I use it in a Librem 15v4, with 32GB of RAM.

It's not only about threats, it's pretty convenient. I do all my dd operations, feeling confident a mistake won't wipe out my HDD. I have a work vm and a personal vm (and many more), and I can share full screen on my work vm knowing that all personal windows are hidden.

I have files and programs organized by vms. I can try installing new applications in a disposable vm knowing well that all their files will be wiped out when I close the vm.

You work in cybersecurity and want more exposure between the host and the guest? You have a very different risk tolerance than I would in your shoes

It's about reducing the size and attack surface of the most-privileged code which runs in the system, e.g. moving code out of the kernel, making hypervisor/VMM smaller, nested VMs, hardware enclaves. This video covers some of the changes over the last decade, including Xen and Bromium, https://youtube.com/watch?v=bNVe2y34dnM

I use vbox regularly on a Linux host, it’s not seamless but it works okay. I have custom built vm images with packer that do things like enable auto login and disable screensaver (these don’t matter on a vm, your host is where they should happen). I don’t need gpu so the vbox drivers suffice, but if I did I would probably consider getting a quadro or something and doing pci pass through (not even sure if vbox supports this)

As a cautionary though, vms are a good boundary but not a comprehensive one. If your threat model includes execution of 0day exploits (malware analysis or browser exploit chains) that can breach hypervisor perimeters you shouldn’t be doing anything sensitive from the host. RDP is better, but iirc there are some case studies of execution on the rdp client.

GPU Passthrough can be solved with LookingGlass (https://looking-glass.io/) if you just want a solve that particular problem. I'm not sure how well it works on a laptop but if you have a dedicated graphics card (e.g. Nvidia) you should theoretically be able to get it working the way you want. I'm sorry for the lack of elegant all-in-one packages. I too wish for an Excalibur of VM solutions.

Related:

Qubes OS: A reasonably secure operating system - https://news.ycombinator.com/item?id=30776103 - March 2022 (97 comments)

Qubes OS 4.1.0 has been released - https://news.ycombinator.com/item?id=30215210 - Feb 2022 (1 comment)

Ask HN: Qubes OS or just separate VMs for separating work and private files? - https://news.ycombinator.com/item?id=29537961 - Dec 2021 (6 comments)

Qubes OS 4.1 RC2 - https://news.ycombinator.com/item?id=29402767 - Dec 2021 (1 comment)

Qubes OS 4.1-rc1 has been released - https://news.ycombinator.com/item?id=28856957 - Oct 2021 (5 comments)

Qubes-Lite with KVM and Wayland - https://news.ycombinator.com/item?id=26378854 - March 2021 (48 comments)

Ask HW: Qubes OS alternative on LXD containers - https://news.ycombinator.com/item?id=25562208 - Dec 2020 (21 comments)

Ask HN: Would it be possible to reimplement Qubes OS but lighter? - https://news.ycombinator.com/item?id=20622850 - Aug 2019 (2 comments)

Joanna Rutkowska leaves Qubes OS, joins Golem - https://news.ycombinator.com/item?id=18300345 - Oct 2018 (68 comments)

Introducing the Qubes U2F Proxy - https://news.ycombinator.com/item?id=17958219 - Sept 2018 (2 comments)

Qubes OS 4.0 has been released - https://news.ycombinator.com/item?id=16699900 - March 2018 (39 comments)

Qubes Air: Generalizing the Qubes Architecture - https://news.ycombinator.com/item?id=16255251 - Jan 2018 (65 comments)

Qubes OS: A reasonably secure operating system - https://news.ycombinator.com/item?id=15734416 - Nov 2017 (144 comments)

Reasonably Secure Computing in the Decentralized World - https://news.ycombinator.com/item?id=15566563 - Oct 2017 (44 comments)

Toward a Reasonably Secure Laptop - https://news.ycombinator.com/item?id=14743238 - July 2017 (100 comments)

“Paranoid Mode” Compromise Recovery on Qubes OS - https://news.ycombinator.com/item?id=14218504 - April 2017 (14 comments)

Tor at the Heart: Qubes OS - https://news.ycombinator.com/item?id=13272076 - Dec 2016 (1 comment)

Qubes OS Begins Commercialization and Community Funding Efforts - https://news.ycombinator.com/item?id=13069615 - Nov 2016 (24 comments)

Qubes OS 3.2 has been released - https://news.ycombinator.com/item?id=12604417 - Sept 2016 (30 comments)

Xen exploitation part 3: XSA-182, Qubes escape - https://news.ycombinator.com/item?id=12232932 - Aug 2016 (5 comments)

Security challenges for the Qubes build process - https://news.ycombinator.com/item?id=11801093 - May 2016 (17 comments)

Qubes OS 3.1 has been released - https://news.ycombinator.com/item?id=11260857 - March 2016 (44 comments)

Qubes OS will ship pre-installed on Purism’s security-focused Librem 13 laptop - https://news.ycombinator.com/item?id=10736516 - Dec 2015 (109 comments)

Finally, a 'Reasonably-Secure' Operating System: Qubes R3 - https://news.ycombinator.com/item?id=10654193 - Dec 2015 (1 comment)

Converting untrusted PDFs into trusted ones: The Qubes Way (2013) - https://news.ycombinator.com/item?id=10538888 - Nov 2015 (5 comments)

Enhancing Qubes with Rumprun unikernels - https://news.ycombinator.com/item?id=10518842 - Nov 2015 (5 comments)

Critical Xen bug in PV memory virtualization code - https://news.ycombinator.com/item?id=10471912 - Oct 2015 (80 comments)

Qubes – Secure Desktop OS Using Security by Compartmentalization - https://news.ycombinator.com/item?id=8428453 - Oct 2014 (49 comments)

Introducing Qubes 1.0 ("a stable and reasonably secure desktop OS") - https://news.ycombinator.com/item?id=4472403 - Sept 2012 (59 comments)

Qubes: an open source OS with strong security for desktop computing - https://news.ycombinator.com/item?id=2645170 - June 2011 (16 comments)

Review: Qubes OS Beta 1 — a new and refreshing approach to system security - https://news.ycombinator.com/item?id=2504274 - May 2011 (1 comment)

* The Linux Security Circus: On GUI isolation* - https://news.ycombinator.com/item?id=2477667 - April 2011 (47 comments)

Qubes Beta 1 has been released (strong desktop security OS) - https://news.ycombinator.com/item?id=2439096 - April 2011 (3 comments)

Qubes Architecture - actual security-oriented OS - https://news.ycombinator.com/item?id=1796384 - Oct 2010 (1 comment)

Open source Qubes OS is ultra secure - https://news.ycombinator.com/item?id=1249857 - April 2010 (7 comments)

Introducing Qubes OS - https://news.ycombinator.com/item?id=1246990 - April 2010 (20 comments)

Using Qubes over a year on my personal laptop, I found 16GB to be too fussy and I constantly had to fiddle with VM RAM sizes. I would recommend 32GB.

NixOS or Guix both allow one to fire up a vm based on a specification very easily, and positively encourage interation. The learning curve is steep but rewarding.

If you just have a need for isolating Windows applications have you tried the Windows Sandbox functionality built-in to Windows 10 Pro and Enterprise version? https://docs.microsoft.com/en-us/windows/security/threat-pro...

Yep, IOMMU support used to be one of those features Intel used for product segmentation, eg. disabling it on the -K overclockable CPUs while leaving it enabled on the counterparts with locked multipliers. Thunderbolt is what forced them to stop playing that game.

https://spectrum-os.org/ at least in its goals and design looks promising in that regard.

I'd bet you dollars to donuts that they're using it to host Whonix nodes for their infrastructure.

If I'm doing real malware execution and analysis, I would one- way transfer the relevant file(s) to my sandbox and disable any backward connectivity before execution, but I still need a reasonably simple way of getting files to (suspicious files, etc) and from (resulting logs, registry changes, pcap, etc) the malware sandbox. Ive kinda solved this already using a number of tools outside my work host, but just in the off situations where this is necessary I want to have a template VM prepped in advance.

It’s very different. Can I open a PDF from Windows into Windows Sandbox with 2 clicks and close with 1 after reading, nearly instantaneously, without noticing the virtualization? Nope. Windows Sandbox is just a fresh VM, almost useless.

If you're on Windows, and you want to download something, install it, see what its all about.

Windows Sandbox starts in like 8 seconds to be usable and is trashed when you close it.

So its far from useless.

But for your usecase, yes it wont work.

Whonix nodes for what?

Knowing what I know of Qubes, and reading that post - I think they mean lots of folks are using it for their workstations. Qubes doesn't really make sense on a server, which is how I'd normally read "infrastructure".

Used to use Qubes OS, quit because I could not figure out how to move files / share files between VMs securely / easily.

Want to try again, but a little concerned that AWS seems to have moved away from Xen.

One of the points that look like a plus to me, was that Qubes OS was based on a widely used, battle tested hypervisor, Xen.

The biggest cloud platforms looks like AWS, Google Cloud Platform (GCP), and Microsoft Azure. I should check if what hypervisors they are using.

Looking for AWS use of Xen and other hypervisors, to see if Xen is still widely used, but could not figure out if Xen will continued to be used, or if AWS is moving from Xen to KVM.

Some web searches that I have made were ...

* doc

https://github.com/awsdocs/amazon-ec2-user-guide-windows/tre...

  ... Nitro-based instance type, such as M5 or C5 ...
  ... instance based on the Xen System, such as M4 or C4 ...

* FAQ

https://aws.amazon.com/ec2/faqs/

  Q. Will AWS continue to invest in its Xen-based hypervisor?
  Yes. ...

  Q. What is the Nitro Hypervisor?
  ... The Nitro Hypervisor is built on core Linux Kernel-based Virtual Machine (KVM) technology ...

> I am not sure how to interpret this. Maybe "Q. Will AWS continue to invest in its Xen-based hypervisor?" is a Marketing / PR way of phrasing something?

* Others

https://brendangregg.com/blog/2021-07-05/computing-performan... https://www.usenix.org/conference/lisa21/presentation/gregg-... https://www.usenix.org/system/files/lisa21_slides_gregg_comp...

  VM Improvements
  #6 VM Xen AWS 2017
  #7 VM AWS Nitro 2017

Google Cloud Platform (GCP) looks like it does not use Xen, and only KVM. I could be wrong.

I cloud only find pages with mention of KVM, and no Xen.

https://cloud.google.com/compute/docs/instances/nested-virtu...

  Compute Engine VMs run on a physical host that has Google's security-hardened, KVM-based hypervisor.

I thought Microsoft Azure used only Hyper-V, but doc page mentions used of KVM. I am not sure how to interpret this.

https://github.com/MicrosoftDocs/azure-docs/blob/main/articl...

  The hypervisors for preparation that are covered in this article are Hyper-V, kernel-based virtual machine (KVM), and VMware.
  KVM
  This section shows you how to use KVM to prepare a RHEL 6 or RHEL 7 distro to upload to Azure.

> Maybe this means that Azure is not using KVM, and only uses Hyper-V, but it can import KVM images?

If all the criminal elements on the internet including the dark web are actually the state, then worrying about device isolation is the least of your worries!

Seeing all those OS logos in the corner of each VM makes me think this is enormously inefficient to run on a desktop/laptop

Like... Fedora and Debian? Why?

So their sysadmins don't get hacked.

VPN providers are an annoyance for "collect it all" types. If they have the ingress servers' wireguard private keys, that annoyance goes away. Note that I use the term "annoyance", not "threat"... they sit in that gray zone where it's an irritant but not worth kidnapping people.

> The iommu device is present on nearly all systems these days, even consumer ones.

Along with the IME device or PSP device, which conveniently get to bypass the iommu.

Finding machines with an iommu and without an IME/PSP/equivalent is remarkably difficult. It's basically modern POWER9, 2013-era Opterons, and one or two chromebook-grade Rockchip devices.

You can do exactly that. Setup a .wsb file that mounts your downloads folder to the VM desktop and disable network.

Open the .wsb (1 click) then open the PDF (1 or 2 clicks). When you're finished close the windows sandbox window and it'll all be gone.

I second this, it's called magic wormhole, if you want the source, croc is a more friendly solution built ontop of magic wormhole. In any event, quick and easy.

>then worrying about device isolation is the least of your worries!

Like your microphone or cam?

What's the video performance in Qubes OS like?

When running with VMs anywhere, I'm constantly aware that moving those pixels around is generally incurring a huge overhead.

Ephemeral server instances?

Using a zero install PDF viewer, the wsb login command and a few lines script file one could also have the PDF automatically open in the Windows sandbox with a double click or using "Open with" from the context menu.

Or operating system and hardware circuit design.

Senseless discussion at that point.

You could piggyback off MS Edge in the sandbox to open the PDF. Provided you don't need any advanced features, and just want to read it.

But yes that is a great idea.

You don't have to.

But I don't really see how it relates to efficiency however since all are VMs running their own kernel. It doesn't change anything that they are different versions or not.

One can also ask for help on the Qubes forum: https://forum.qubes-os.org.

I'm talking more about the sheer quantity of operating systems running in parallel on one computer.

I can't find information online. Does xen really has it's own, written from scratch kernel or is it based on some other os?

My super-short introduction: https://forum.qubes-os.org/t/newbie-tutorial-s/9349/4

You simply do everything in virtual machines. Here is why: https://forum.qubes-os.org/t/how-to-pitch-qubes-os/4499/15

IME is disabled and neutralized on my Librem 15, which runs Qubes flawlessly.

> I'd like the ability to drop files to a VM from another VM, like shared folders in Workstation.

https://www.qubes-os.org/doc/how-to-copy-and-move-files/

You only run as many as you want, depending on your compartmentalization strategy. Also, they are typically not all taking 100% CPU simultaneously.

> Used to use Qubes OS, quit because I could not figure out how to move files / share files between VMs securely / easily.

Is this what your're looking for? https://www.qubes-os.org/doc/how-to-copy-and-move-files/

Completely it's own from scratch. It's a very simple, relative to full Linux or other OS, kernel/hypervisor. It's built to load what it calls the Dom0 system, which it gives a communication channel to to start up other virtual systems that it calls DomU. This in theory lets you use any OS as the Dom0 for initializing everything (Linux, *BSD, even Windows I think) as long as there's some kind of support for that communications channel to tell the Xen kernel to start up another system.

https://wiki.xenproject.org/wiki/Xen_Project_Software_Overvi...

Not good at all. Playing non-HD video incurs significant CPU overhead, and HD is out of the question on all but the most powerful processors.

how does it cope with multi-monitor setups?

I put 64gb ram in a librem 14. Cost around $200. The only thing I feel is lack of GPU. The hardware all works great. (The librem build itself is meh, but it all works perfectly with qubes without any tinkering.)

I was more thinking of running a portable pdf viewer such as Foxit Reader Portable from a folder mounted read only in the sandbox.

An external screen works fine for me. It's just xfce (or KDE).

My i7-10710U runs HD videos fine.

PSA: wormhole.app and magic wormhole are not the same

If you want to use the well known magic wormhole then visit the repo for instructions: https://github.com/magic-wormhole/magic-wormhole

The current supported version is a python cli app. A rust version is being developed, but last I checked was not considered ready.