←back to thread

Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

(twitter.com)
229 points modinfo | 4 comments | 30 Jun 24 01:56 UTC | HN request time: 0.653s | source
Show context
simonw ◴[30 Jun 24 06:52 UTC] No.40835549[source]▶
If this is the API that Google are going with here:

    const model = await window.ai.createTextSession();
    const result = await model.prompt("3 names for a pet pelican");
There's a VERY obvious flaw: is there really no way to specify the model to use?

Are we expecting that Gemini Nano will be the one true model, forever supported by this API baked into the world's most popular browser?

Given the rate at which models are improving that would be ludicrous. But... if the browser model is being invisibly upgraded, how are we supposed to test out prompts and expect them to continue working without modifications against whatever future versions of the bundled model show up?

Something like this would at least give us a fighting chance:

    const supportedModels = await window.ai.getSupportedModels();
    if (supportedModels.includes("gemini-nano:0.4")) {
        const model = await window.ai.createTextSession("gemini-nano:0.4");
        // ...
replies(7): >>40835678 #>>40835703 #>>40835717 #>>40835757 #>>40836197 #>>40836971 #>>40843533 #
1. luke-stanley ◴[30 Jun 24 07:47 UTC] No.40835717[source]▶
Presumably something like model.includes("gemini-nano:0.4") could work?
replies(1): >>40836178 #
2. damacaner ◴[30 Jun 24 09:57 UTC] No.40836178[source]▶
can we make everything constant like C# does please

Models.GeminiNano04

boom

replies(2): >>40837121 #>>40843547 #
3. jitl ◴[30 Jun 24 13:40 UTC] No.40837121[source]▶
What’s the point in JavaScript? At the end of the day that’s still equivalent to Models[“GeminiNano04”]

In C# you can’t compile a reference to Models.Potato04 unless Potato04 exists. In JS it’s perfectly legal to have code that references non-existant properties, so there’s no real developer ergonomics benefit here.

On the contrary, code like `ai.createTextSession(“Potato:4”)` can throw an error like “Model Potato:4 doesn’t exist, try Potato:1”, whereas `ai.createTextSession(ai.Models.Potato04)` can only throw an error like “undefined is not a Model. Pass a string here”.

Or you can make ai.Models a special object that throws when undefined properties are accessed, but then it’s annoying to write code that sniffs out which models are available.

4. langcss ◴[01 Jul 24 08:01 UTC] No.40843547[source]▶
That would be like making a constant for every nuget package/version tuple: unworkable because new versions and packages come out all the time.

Or making constants for every device manufacturer you can connect to via web Bluetooth.