←back to thread

Mercury: Ultra-fast language models based on diffusion

(arxiv.org)
566 points PaulHoule | 1 comments | 07 Jul 25 12:31 UTC | HN request time: 0.282s | source
Show context
pmxi ◴[07 Jul 25 14:19 UTC] No.44490644[source]▶
This is cool. I think faster models can unlock entirely new usage paradigms, like how faster search enables incremental search.
replies(1): >>44496718 #
1. physix ◴[08 Jul 25 03:26 UTC] No.44496718[source]▶
I was curious to know the statistics on the mentions of various programming languages on HN over the years, so I got me a copy of all HN comments from a BigTable public source. But now I need to interpret each comment and so what I need is a semantic grep. The easiest would be to prompt an LLM.

Comments are pretty short, but there are many millions of them. So getting high throughput at minimum cost is key.

I'm hoping that Inception might be able to churn through this quickly.

If you folks have other ideas or suggestions, what might also work, I'd love to hear them!

The idea is having a semgrep command line tool. If latencies are dropping dramatically, it might be feasible.