←back to thread

Neural audio codecs: how to get audio into LLMs

(kyutai.org)
425 points karimf | 2 comments | 21 Oct 25 12:55 UTC | HN request time: 0s | source
Show context
miki123211 ◴[21 Oct 25 14:28 UTC] No.45656279[source]▶
> Try asking any of them “Am I speaking in a low voice or a high voice?” in a high-pitched voice, and they won’t be able to tell you.

I wonder how much of that is LLMs being bad, and how much is LLMs being (over) aligned not to do it.

AFAIK, Chat GPT Voice mode had to have a lot of safeguards put on it to prevent music generation, accent matching (if you sound Indian, it shouldn't also sound Indian), and assuming ethnicity / biasing based on accents.

It doesn't seem that impossible to me that some of these behaviors have been aligned out of these models out of an abundance of caution.

replies(7): >>45656408 #>>45656467 #>>45656667 #>>45657021 #>>45657291 #>>45658995 #>>45665432 #
1. bongodongobob ◴[21 Oct 25 17:46 UTC] No.45658995[source]▶
Hmm, the last time I played with GPT voice mode it was able to do all kinds of different accents.
replies(1): >>45666270 #
2. phrotoma ◴[22 Oct 25 08:28 UTC] No.45666270[source]▶
Like others I noticed this capability was interfered with in some way. I had fun getting it to speak to me in a cheesy over-the-top Bostonian accent early on, then one day when I tried to demonstrate for a friend it interrupted itself mid-sentence, literally one voice speaking over the other truncated voice, saying something like "I'm sorry I can't mimic voices".

It seemed like they had one model monitoring the output of another model and then cutting it off when it crossed some line.