←back to thread

Neural audio codecs: how to get audio into LLMs

(kyutai.org)
425 points karimf | 2 comments | 21 Oct 25 12:55 UTC | HN request time: 0s | source
Show context
miki123211 ◴[21 Oct 25 14:28 UTC] No.45656279[source]▶
> Try asking any of them “Am I speaking in a low voice or a high voice?” in a high-pitched voice, and they won’t be able to tell you.

I wonder how much of that is LLMs being bad, and how much is LLMs being (over) aligned not to do it.

AFAIK, Chat GPT Voice mode had to have a lot of safeguards put on it to prevent music generation, accent matching (if you sound Indian, it shouldn't also sound Indian), and assuming ethnicity / biasing based on accents.

It doesn't seem that impossible to me that some of these behaviors have been aligned out of these models out of an abundance of caution.

replies(7): >>45656408 #>>45656467 #>>45656667 #>>45657021 #>>45657291 #>>45658995 #>>45665432 #
tsol ◴[21 Oct 25 15:00 UTC] No.45656667[source]▶
Did they respond differently depending on what race they thought you were? I'm surprised they would even do that honestly. I thought they were trained on text conversations which presumably wouldn't have any of that to learn from.
replies(4): >>45656799 #>>45656985 #>>45657478 #>>45664768 #
OisinMoran ◴[21 Oct 25 15:10 UTC] No.45656799[source]▶
You can often tell where someone is from from text alone! There are plenty of idiosyncrasies even in how different English speaking countries use the language.
replies(2): >>45656828 #>>45657486 #
fragmede ◴[21 Oct 25 16:05 UTC] No.45657486{3}[source]▶
Like, what do you mean? Are there, like, particular mannerisms that people from some regions that are hella unique to those regions?
replies(5): >>45657662 #>>45657679 #>>45660771 #>>45660775 #>>45664749 #
1. xwolfi ◴[22 Oct 25 03:56 UTC] No.45664749{4}[source]▶
All my Indian colleagues say "I agree with the same", this "the same" turn of phrase was so strange to me I had to ask (I'm French, so I have my own silly quirks, like I forget non-vocal plural(s<-- see, often I don't write that s)). They told me it was like that in Hindi so they just reproduce the pattern and it's grammatically acceptable.

For French people like me, false friends are immediately noticeable: for instance, "actually" to mean "now" instead of "in fact".

replies(1): >>45677405 #
2. Xmd5a ◴[23 Oct 25 02:07 UTC] No.45677405[source]▶
>like I forget non-vocal plural(s<-- see, often I don't write that s)).

c'est infernal. infernal