←back to thread

Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

(twitter.com)
229 points modinfo | 7 comments | 30 Jun 24 01:56 UTC | HN request time: 0.634s | source | bottom
Show context
oefrha ◴[30 Jun 24 02:57 UTC] No.40834836[source]▶
See

https://developer.chrome.com/docs/ai/built-in

https://github.com/jeasonstudio/chrome-ai

I can’t seem to find public documentation for the API with a cursory search, so https://github.com/jeasonstudio/chrome-ai/blob/ec9e334253713... might be the best documentation (other than directly inspecting the window.ai object in console) at the moment.

It’s not really clear if the Gemini Nano here is Nano-1 (1.8B) or Nano-2 (3.25B) or selected based on device.

replies(2): >>40834950 #>>40835017 #
1. mikeqq2024 ◴[30 Jun 24 03:48 UTC] No.40835017[source]▶
1.8B/3.25B is still too much for edge devices. Ideally tens or hundreds mega would be ok. Is there an option to change the builtin Gemini Nano to other smaller models?

By the way, haven't touch the lastest JS code for a while, what does this new syntax mean: "import { chromeai } "

Also not get the textStream code: for await (const textPart of textStream) { result = textPart; } does result get override for each loop step?

replies(3): >>40835066 #>>40835093 #>>40835188 #
2. dchest ◴[30 Jun 24 04:03 UTC] No.40835066[source]▶
That's just some supplemental code. The actual API is:

    const session = await window.ai.createTextSession()
    const outputText = await session.prompt(inputText)
That's all there is for now (createGenericSession does the same at this time, and there are canCreateTextSession/canCreateGenericSession).
replies(1): >>40865237 #
3. oefrha ◴[30 Jun 24 04:13 UTC] No.40835093[source]▶
These are 4-bit models.
replies(1): >>40836074 #
4. kevindamm ◴[30 Jun 24 04:40 UTC] No.40835188[source]▶
`import { chromeai } from ...` is doing destructuring of the exported symbols of the module being imported. So, here it only imports the variable or type or function named chromeai.
replies(1): >>40835243 #
5. ◴[30 Jun 24 04:59 UTC] No.40835243[source]▶
6. zecg ◴[30 Jun 24 09:30 UTC] No.40836074[source]▶
Made by a by a two-bit company that...
7. dchest ◴[03 Jul 24 12:19 UTC] No.40865237[source]▶
Update: there's actually quite a lot more: https://github.com/explainers-by-googlers/prompt-api