←back to thread

Show HN: I open-sourced my AI toy company that runs on ESP32 and OpenAI realtime

(github.com)
177 points akadeb | 3 comments | 22 Apr 25 14:10 UTC | HN request time: 0.772s | source

Hi HN! Last year the project I launched here got a lot of good feedback on creating speech to speech AI on the ESP32. Recently I revamped the whole stack, iterated on that feedback and made our project fully open-source—all of the client, hardware, firmware code.

This Github repo turns an ESP32-S3 into a realtime AI speech companion using the OpenAI Realtime API, Arduino WebSockets, Deno Edge Functions, and a full-stack web interface. You can talk to your own custom AI character, and it responds instantly.

I couldn't find a resource that helped set up a reliable, secure websocket (WSS) AI speech to speech service. While there are several useful Text-To-Speech (TTS) and Speech-To-Text (STT) repos out there, I believe none gets Speech-To-Speech right. OpenAI launched an embedded-repo late last year which sets up WebRTC with ESP-IDF. However, it's not beginner friendly and doesn't have a server side component for business logic.

This repo is an attempt at solving the above pains and creating a great speech to speech experience on Arduino with Secure Websockets using Edge Servers (with Deno/Supabase Edge Functions) for fast global connectivity and low latency.

Show context
empath75 ◴[22 Apr 25 15:26 UTC] No.43763312[source]▶
When someone figures this out, it's going to be a multi billion dollar company, but the safety concerns for actually putting something like this into the hands of children are unbelievable.
replies(3): >>43763354 #>>43763562 #>>43763936 #
mithr ◴[22 Apr 25 15:50 UTC] No.43763562[source]▶
This. The idea is super cool in theory! But given how these sort of things work today, having a toy that can have an independent conversation with a kid and that, despite the best intentions of the prompt writer, isn't guaranteed to stay within its "sandbox", is terrifying enough to probably not be worth the risk.

IMO this is only exacerbated by how little children (who are the presumably the target audience for stuffed animals that talk) often don't follow "normal" patterns of conversation or topics, so it feels like it'd be hard to accurately simulate/test ways in which unexpected & undesirable responses could come out.

replies(1): >>43763975 #
conductr ◴[22 Apr 25 16:34 UTC] No.43763975[source]▶
I'm trying to use my imagination, but what exactly is the fear? Perhaps the AI will explain where baby's come from in graphic detail before the parent is ready to have that conversation or something similar? Or, for us in US, maybe it tells your kid they should wear a bullet proof vest to pre-K instead of bringing a stuffy for naptime?

Essentially, telling kids the truth before they're ready and without typical parental censorship? Or is there some other fear, like the AI will get compromised by a pedo and he'll talk your kid into who knows what? Or similar for "fill in state actor" using mind control on your kid (which, honestly, I feel like is normalized even for adults; eg. Fox News, etc., again US-centric)

replies(3): >>43764156 #>>43764946 #>>43765512 #
3np ◴[22 Apr 25 19:36 UTC] No.43765512[source]▶
How about encouraging self-harm, even murder and suicide?

https://www.npr.org/2024/12/10/nx-s1-5222574/kids-character-...

https://apnews.com/article/chatbot-ai-lawsuit-suicide-teen-a...

https://www.euronews.com/next/2023/03/31/man-ends-his-life-a...

replies(1): >>43765896 #
1. conductr ◴[22 Apr 25 20:22 UTC] No.43765896[source]▶
Can this not occur on Youtube/Roblox and other places where kids using tablets go? Mass generalizations about what I observe -> I don't see why/how parents do the mental gymnastics that tablets are acceptable but AI is to be feared. There's always going to be articles like this, it's a big world everything will have a dark side if you search for it. It's life. [Actually, I think a lot of parents are willing to accept/ignore the risks because tablets offer too great of a service. This type of AI simply won't entertain/babysit a kid long enough for parents to give into it.]

I have a 6 year old FWIW, I'm not some childless ignoramus I just do my risk calcs differently and view it as my job to oversee their use of a device like this. I wouldn't fear it outright because of what could happen. If I took that stance, my kid would never have any experiences at all.

Can't play baseball, I read a story where kid got hit by a bat. Can't travel to Mexico, cartels are in the news again. Home school it is, because shootings. And so on.

replies(1): >>43766987 #
2. 3np ◴[22 Apr 25 22:42 UTC] No.43766987[source]▶
A 6yo can not meaningfully give informed consent to ToSs or privacy polies of YouTube and Roblox so even supervised is ethically problematic depending on how it's done. Unsupervised is obviously not safe and I do not see anyone here arguing that.
replies(1): >>43769170 #
3. conductr ◴[23 Apr 25 06:18 UTC] No.43769170[source]▶
I don't think that argument needs to be made here, I mentioned it because it's something I observe daily in the real world. I talk to parents who let their kids use these things, I inquire about their reasons for doing so and their level of oversight. It's something I've personally taken an interest in as a parent myself who has a no tolerance policy towards it; I like to know other people's justifications for allowing it. Many of them do not supervise the use BTW. Even people I consider great parents otherwise, they may setup some parental control stuff initially but then the kid is off with their device in another room.

When it comes to privacy policies and ToS, I think a 6yo is reading into it just as much as their parent does. And by that I mean just looking for the [Agree] button.