←back to thread

Show HN: Lightpanda, an open-source headless browser in Zig

(github.com)
319 points fbouvier | 2 comments | 24 Jan 25 22:15 UTC | HN request time: 0.428s | source

We’re Francis and Pierre, and we're excited to share Lightpanda (https://lightpanda.io), an open-source headless browser we’ve been building for the past 2 years from scratch in Zig (not dependent on Chromium or Firefox). It’s a faster and lighter alternative for headless operations without any graphical rendering.

Why start over? We’ve worked a lot with Chrome headless at our previous company, scraping millions of web pages per day. While it’s powerful, it’s also heavy on CPU and memory usage. For scraping at scale, building AI agents, or automating websites, the overheads are high. So we asked ourselves: what if we built a browser that only did what’s absolutely necessary for headless automation?

Our browser is made of the following main components:

- an HTTP loader

- an HTML parser and DOM tree (based on Netsurf libs)

- a Javascript runtime (v8)

- partial web APIs support (currently DOM and XHR/Fetch)

- and a CDP (Chrome Debug Protocol) server to allow plug & play connection with existing scripts (Puppeteer, Playwright, etc).

The main idea is to avoid any graphical rendering and just work with data manipulation, which in our experience covers a wide range of headless use cases (excluding some, like screenshot generation).

In our current test case Lightpanda is roughly 10x faster than Chrome headless while using 10x less memory.

It's a work in progress, there are hundreds of Web APIs, and for now we just support some of them. It's a beta version, so expect most websites to fail or crash. The plan is to increase coverage over time.

We chose Zig for its seamless integration with C libs and its comptime feature that allow us to generate bi-directional Native to JS APIs (see our zig-js-runtime lib https://github.com/lightpanda-io/zig-js-runtime). And of course for its performance :)

As a company, our business model is based on a Managed Cloud, browser as a service. Currently, this is primarily powered by Chrome, but as we integrate more web APIs it will gradually transition to Lightpanda.

We would love to hear your thoughts and feedback. Where should we focus our efforts next to support your use cases?

Show context
m3kw9 ◴[24 Jan 25 16:19 UTC] No.42814620[source]▶
How does this work because the browser needs to render a page and the vision model needs to know where a button is, so it still needs to see an image. How does headless make it easier?
replies(1): >>42814714 #
katiehallett ◴[24 Jan 25 16:29 UTC] No.42814714[source]▶
Headless mode skips the visual rendering meant for humans, but the DOM structure and layout still exist, allowing the model to parse elements programmatically (e.g. button locations). Instead of 'seeing' an image, the model interacts with the page's underlying structure, which is faster and more efficient. Our browser removes the rendering engine as well, so it won't handle 100% of automation use cases, but it's also what allows us to be faster and lighter than Chrome in headless mode.
replies(2): >>42815232 #>>42816611 #
wiradikusuma ◴[24 Jan 25 17:16 UTC] No.42815232[source]▶
But what if the human programmer needs to visually verify that their code works by eyeballing which element got selected, etc?
replies(3): >>42815333 #>>42815363 #>>42815366 #
1. fbouvier ◴[24 Jan 25 17:26 UTC] No.42815333[source]▶
You're right, the debugging part is a good use case for graphical rendering in a headless environment.

I see it as a build time/runtime question. At build (dev) time I want to have a graphical response (debugging, computer vision, etc.). And then, when the script is ready, I can use Lightpanda at runtime as a lightweight alternative.

replies(1): >>42818211 #
2. codetrotter ◴[25 Jan 25 00:15 UTC] No.42818211[source]▶
I was doing a personal side project for a while where I was trying to make my own little Wayback Machine-alike. Mine was very rudimentary, built on top of Firefox and WebDriver plus Squid proxy.

For debugging purposes you could have your headless browser function as a HTTP Proxy Server, maybe? And in your headless browser you could capture a static snapshot of the DOM after your JavaScript runtime has executed the scripts for the page. Similar to how the archive.today guy serves static snapshots of websites. And then developers using your headless browser could point their Firefox or Chrome browser to the HTTP Proxy server hosted by your headless browser program, in order to get a static snapshot view of what the DOM is like after your headless browser has executed JavaScript from the page. And then Firefox or Chrome will render static HTML view of what the page looked like to your headless browser, that the developer can inspect to make decisions about further interactions with the page. As a tool for debugging.