GGML – AI at the Edge

I've always thought on the edge to be IoT type stuff. So running on embedded devices. But maybe that not the case?

Like any new term the (mis)usage broadens the meaning over time until it either it's widely known, it's unfashionable, or most likely; it becomes so broad as to be meaningless and hence it achieves buzzword apotheosis.

My old job title had "edge" in it, and I still don't know what it's supposed to mean, although "not cloud" is a good approximation.

"Edge computing" is a pretty vague term, and can encompass anything from a 8MHz ARM core that can barely talk compliant BLE, all the way to a multi-thousand dollar setup on something like a self-checkout machine, which may have more compute available than your average laptop. In that range are home assistants, which normally have some basic ML for wake word detection, and then send the next bit of audio to the cloud with a more advanced model for full speech-to-text (and response)

Sounds like your job had a lot of velocity with lateral tragmorphicity in Q1, just in time for staff engineer optimization!

Nicely done. Here is ~$50 worth of stock.

Here it means more 'on your own device' rather than 'in the cloud'

You could consider that the real edge, whereas edge computing often means 'at the edge of the cloud' i.e. local CDN node

edge brings compute close to where data is generated, cloud brings data to compute.

even processing something in a web browser is called edge. i guess due to this impression the industry is moving towards "on-device"