←back to thread

Show HN: Index – New Open Source browser agent

(github.com)
98 points skull8888888 | 5 comments | 23 Apr 25 16:11 UTC | HN request time: 0s | source

Hey HN, Robert from Laminar (lmnr.ai) here.

We built Index - new SOTA Open Source browser agent.

It reached 92% on WebVoyager with Claude 3.7 (extended thinking). o1 was used as a judge, also we manually double checked the judge.

At the core is same old idea - run simple JS script in the browser to identify interactable elements -> draw bounding boxes around them on a screenshot of a browser window -> feed it to the LLM.

What made Index so good:

1. We essentially created browser agent observability. We patched Playwright to record the entire browser session while the agent operates, simultaneously tracing all agent steps and LLM calls. Then we synchronized everything in the UI, creating an unparalleled debugging experience. This allowed us to pinpoint exactly where the agent fails by seeing what it "sees" in session replay alongside execution traces.

2. Our detection script is simple but extremely good. It's carefully crafted via trial and error. We also employed CV and OCR.

3. Agent is very simple, literally just a while loop. All power comes from carefully crafted prompt and ton of eval runs.

Index is a simple python package. It also comes with a beautiful CLI.

pip install lmnr-index

playwright install chromium

index run

We've recently added o4-mini, Gemini 2.5 Pro and Flash. Pro is extremely good and fast. Give it a try via CLI.

You can also use index via serverless API. (https://docs.lmnr.ai/index-agent/api/getting-started)

Or via chat UI - https://lmnr.ai/chat.

To learn more about browser agent observability and evals check out open-source repo (https://github.com/lmnr-ai/lmnr) and our docs (https://docs.lmnr.ai/tracing/browser-agent-observability).

Show context
androng ◴[23 Apr 25 22:56 UTC] No.43777550[source]▶
Can it actually do something difficult like apply for jobs? So far I know of five or so websites that claim they can apply to jobs for you like sonara.ai and usemassive.com and Skyvern AI but when you try to actually use them all they can do is the one-page job applications and not the much more common Workday 10-page job applications with annoying "create an account" and annoying questions like "Do you have any relatives that work at Sony" and annoying "fill out all your work experience" where you have to click 50 times for one application. That's like half of all job applications. https://jobs.spectrum.com/job/-/-/4673/76746020384?utm_sourc...
replies(3): >>43777578 #>>43777698 #>>43777893 #
esafak ◴[23 Apr 25 23:14 UTC] No.43777698[source]▶
I consider using bad hiring software like that a red flag, and suggestive of other things the company must be doing wrong too. I noped out whenever I saw Taleo.
replies(1): >>43778870 #
1. globalnode ◴[24 Apr 25 02:54 UTC] No.43778870[source]▶
All big successful companies do "something" wrong, thats how they make money. Steal your OSS, not pay taxes, avoid overtime payments, low wages, outsource slavery, destroy the environment, gaslight while they steal your data and subject millions to dark patterns of advertising and marketing, screw over suppliers, intentionally sew discord as a distraction. The list goes on. To me the bigger the company the bigger the red flag.
replies(1): >>43780093 #
2. tomdekan ◴[24 Apr 25 07:26 UTC] No.43780093[source]▶
That’s a very cynical view.

Do the biggest companies not create the most value for the world?

Consider this. If the most successful companies are simply cheating customers, then most consumers are stupid; handing offer their hard-earned money for bad deals and to be exploited.

But most people are not stupid, and most people highly value their money. So, they only buy something because they want what the seller is offering even more than their money. This means that companies create great value because they offer something that people really want.

replies(1): >>43788535 #
3. noooooooph ◴[24 Apr 25 23:10 UTC] No.43788535[source]▶
This assumes there are compelling alternatives in the market that I can choose from. In reality, there are only a few entrenched players in any established market that work hard to limit competition. So yes, even if I'd like to choose not to hand over my hard-earned money to Evil Corp #93, I can only do the "stupid" thing and watch myself and my environment get exploited.
replies(2): >>43791098 #>>43791158 #
4. tomdekan ◴[25 Apr 25 07:22 UTC] No.43791098{3}[source]▶
I disagree with the general comment that “any established market” has “only a few entrenched players”. I’d say that most markets provide compelling alternatives. Where they don’t yet, the product is either a commodity, or there is an opportunity for a new business to serve the customer!

But let’s say your point is true. How do those players become entrenched? I’d say it’s from providing great value.

5. tomdekan ◴[25 Apr 25 07:32 UTC] No.43791158{3}[source]▶
Also, I don’t think my previous comment does assume that there are compelling alternatives.

A person always has a choice not to spend their money. Even if they need expensive healthcare, they can choose not to buy it. By buying the product, they want the service more than their money.

They might think that the price is too high, but prices are a function of market forces.

It doesn’t make sense to me that a person can say they feel exploited because they have voluntarily chosen to buy at a particular price. They probably want to pay less, and might feel that the consumer surplus is low, but they still value the service more than their money. That isn’t exploitation to me.