←back to thread

Show HN: Lightpanda, an open-source headless browser in Zig

(github.com)
319 points fbouvier | 9 comments | 24 Jan 25 22:15 UTC | HN request time: 0.621s | source | bottom

We’re Francis and Pierre, and we're excited to share Lightpanda (https://lightpanda.io), an open-source headless browser we’ve been building for the past 2 years from scratch in Zig (not dependent on Chromium or Firefox). It’s a faster and lighter alternative for headless operations without any graphical rendering.

Why start over? We’ve worked a lot with Chrome headless at our previous company, scraping millions of web pages per day. While it’s powerful, it’s also heavy on CPU and memory usage. For scraping at scale, building AI agents, or automating websites, the overheads are high. So we asked ourselves: what if we built a browser that only did what’s absolutely necessary for headless automation?

Our browser is made of the following main components:

- an HTTP loader

- an HTML parser and DOM tree (based on Netsurf libs)

- a Javascript runtime (v8)

- partial web APIs support (currently DOM and XHR/Fetch)

- and a CDP (Chrome Debug Protocol) server to allow plug & play connection with existing scripts (Puppeteer, Playwright, etc).

The main idea is to avoid any graphical rendering and just work with data manipulation, which in our experience covers a wide range of headless use cases (excluding some, like screenshot generation).

In our current test case Lightpanda is roughly 10x faster than Chrome headless while using 10x less memory.

It's a work in progress, there are hundreds of Web APIs, and for now we just support some of them. It's a beta version, so expect most websites to fail or crash. The plan is to increase coverage over time.

We chose Zig for its seamless integration with C libs and its comptime feature that allow us to generate bi-directional Native to JS APIs (see our zig-js-runtime lib https://github.com/lightpanda-io/zig-js-runtime). And of course for its performance :)

As a company, our business model is based on a Managed Cloud, browser as a service. Currently, this is primarily powered by Chrome, but as we integrate more web APIs it will gradually transition to Lightpanda.

We would love to hear your thoughts and feedback. Where should we focus our efforts next to support your use cases?

Show context
frankgrecojr ◴[24 Jan 25 20:35 UTC] No.42816664[source]▶
The hello world example does not work. In fact, no website I've tried works. It's usually always panics. For the example in the readme, the errors are:

```

./lightpanda-aarch64-macos --host 127.0.0.1 --port 9222

info(websocket): starting blocking worker to listen on 127.0.0.1:9222

info(server): accepting new conn...

info(server): client connected

info(browser): GET https://wikipedia.com/ 200

info(browser): fetch https://wikipedia.com/portal/wikipedia.org/assets/js/index-2...: http.Status.ok

info(browser): eval script portal/wikipedia.org/assets/js/index-24c3e2ca18.js: ReferenceError: location is not defined

info(browser): fetch https://wikipedia.com/portal/wikipedia.org/assets/js/gt-ie9-...: http.Status.ok

error(events): event handler error: error.JSExecCallback

info(events): event handler error try catch: TypeError: Cannot read properties of undefined (reading 'length')

info(server): close cmd, closing conn...

info(server): accepting new conn...

thread 5274880 panic: attempt to use null value

zsh: abort ./lightpanda-aarch64-macos --host 127.0.0.1 --port 9222

```

replies(3): >>42818479 #>>42819267 #>>42820633 #
1. zelcon ◴[25 Jan 25 03:40 UTC] No.42819267[source]▶
That's Zig for you. A ``modern'' systems programming language with no borrow checker or even RAII.
replies(4): >>42819795 #>>42819912 #>>42820953 #>>42821221 #
2. hansvm ◴[25 Jan 25 05:49 UTC] No.42819795[source]▶
Those statements are mostly true and also worth talking about, but they're not pertinent to that error (remotely provided JS not behaving correctly), or the eventual crash (which you'd cause exactly the same way for the same reason in Rust with a .unwrap() call).
replies(2): >>42819951 #>>42820734 #
3. igorguerrero ◴[25 Jan 25 06:33 UTC] No.42819912[source]▶
You could build the same thing in Rust and have the same exact issue.
4. jbggs ◴[25 Jan 25 06:48 UTC] No.42819951[source]▶
you shouldn't be unwrapping, error cases should be properly handled. users shouldn't see null dereference errors without any context, even in cli tools...
replies(1): >>42825474 #
5. IshKebab ◴[25 Jan 25 10:14 UTC] No.42820734[source]▶
Not exactly the same. `.unwrap()` will never lead to UB, but this can in Zig in release mode.

Also `unwrap()`s are a lot more obvious than just a ?. Dangerous operations should require more ceremony than safe ones. Surprising to see Zig make such a mistake.

replies(1): >>42825560 #
6. steeve ◴[25 Jan 25 11:12 UTC] No.42820953[source]▶
That has absolutely nothing to do with RAII or safety…
7. audunw ◴[25 Jan 25 12:25 UTC] No.42821221[source]▶
If that kind of stuff is always preferable, the nobody would use C over C++, yet to this day many projects still do. Borrow checking isn’t free. It’s a trade-off.

I mean, you could say Rust isn’t a modern language because it doesn’t use garbage collection. But it’s a nonsensical statement. Different languages serve different purposes.

Besides, Zig is focusing a lot more on heavily integrating testing, debug modes, fuzzing, etc. in the compiler itself, which when put together will catch almost all of the bugs a borrow checker catches, but also a whole ton of other classes of bugs that Rust doesn’t have compile time checks for.

I would probably still pick Rust in cases where it’s absolutely critical to avoid bugs that compromise security.

But this project isn’t that kind of project. I’d imagine that the super fast compile times and rapid iteration that Zig provides is much more useful here.

8. hansvm ◴[25 Jan 25 22:19 UTC] No.42825474{3}[source]▶
That too, as a general coding pattern. I was commenting on the criticism of Zig as a sub-par system's language though, contrasting with a language most people with that opinion seem to like.
9. hansvm ◴[25 Jan 25 22:29 UTC] No.42825560{3}[source]▶
> UB vs "safe" panic

Yes, it's not exactly the same if you compile in ReleaseFast instead of ReleaseSafe. Both are bad though, and I'd tend to blame the coding pattern for the observed behavior rather than quibble about which unacceptable scenario is worse.

I see people adopting forced null unwrapping for dumb reasons all the time. For the remaining reasons, do you have a sense of what the language deficiencies are which make that feature helpful? I see it for somewhat sane reasons when crossing language boundaries. Does anything else stand out?

> ceremony

Yes. Thankfully ".?" is greppable, but I wouldn't mind more ceremony there, and less for `try` coding patterns.