Most active commenters

    ←back to thread

    Neural audio codecs: how to get audio into LLMs

    (kyutai.org)
    425 points karimf | 16 comments | 21 Oct 25 12:55 UTC | HN request time: 0.001s | source | bottom
    Show context
    miki123211 ◴[21 Oct 25 14:28 UTC] No.45656279[source]▶
    > Try asking any of them “Am I speaking in a low voice or a high voice?” in a high-pitched voice, and they won’t be able to tell you.

    I wonder how much of that is LLMs being bad, and how much is LLMs being (over) aligned not to do it.

    AFAIK, Chat GPT Voice mode had to have a lot of safeguards put on it to prevent music generation, accent matching (if you sound Indian, it shouldn't also sound Indian), and assuming ethnicity / biasing based on accents.

    It doesn't seem that impossible to me that some of these behaviors have been aligned out of these models out of an abundance of caution.

    replies(7): >>45656408 #>>45656467 #>>45656667 #>>45657021 #>>45657291 #>>45658995 #>>45665432 #
    1. tsol ◴[21 Oct 25 15:00 UTC] No.45656667[source]▶
    Did they respond differently depending on what race they thought you were? I'm surprised they would even do that honestly. I thought they were trained on text conversations which presumably wouldn't have any of that to learn from.
    replies(4): >>45656799 #>>45656985 #>>45657478 #>>45664768 #
    2. OisinMoran ◴[21 Oct 25 15:10 UTC] No.45656799[source]▶
    You can often tell where someone is from from text alone! There are plenty of idiosyncrasies even in how different English speaking countries use the language.
    replies(2): >>45656828 #>>45657486 #
    3. anotherhue ◴[21 Oct 25 15:12 UTC] No.45656828[source]▶
    Ah stop
    4. thwarted ◴[21 Oct 25 15:25 UTC] No.45656985[source]▶
    If it did, it responded based on the accent it picked up on not race, because race and accent are orthogonal, correlation does not imply causation.
    replies(1): >>45659653 #
    5. j45 ◴[21 Oct 25 16:05 UTC] No.45657478[source]▶
    There are subtle differences in language where two groups can be speaking English and one is having a completely different conversation without saying much.
    replies(1): >>45659642 #
    6. fragmede ◴[21 Oct 25 16:05 UTC] No.45657486[source]▶
    Like, what do you mean? Are there, like, particular mannerisms that people from some regions that are hella unique to those regions?
    replies(5): >>45657662 #>>45657679 #>>45660771 #>>45660775 #>>45664749 #
    7. robotresearcher ◴[21 Oct 25 16:19 UTC] No.45657662{3}[source]▶
    I say old chap, what colour are your mummy’s wellies?
    8. ctxc ◴[21 Oct 25 16:20 UTC] No.45657679{3}[source]▶
    Clever!
    9. dotancohen ◴[21 Oct 25 18:27 UTC] No.45659642[source]▶
    This is quite the reason my wife evolved into my ex-wife.
    10. dotancohen ◴[21 Oct 25 18:28 UTC] No.45659653[source]▶
    Are denying that race and accent are highly correlated?
    replies(1): >>45663273 #
    11. ◴[21 Oct 25 19:48 UTC] No.45660771{3}[source]▶
    12. ElevenLathe ◴[21 Oct 25 19:48 UTC] No.45660775{3}[source]▶
    You betcha!
    13. thwarted ◴[21 Oct 25 23:52 UTC] No.45663273{3}[source]▶
    No, I'm saying that it is more meaningful to use what is directly derived rather than what is an indirect assumption. There is already issues with people erroneously considering whatever LLMs output as truth, the last thing anyone needs is an LLM claiming someone like Idris Elba is a white Briton because of his accent. We don't need automated phrenology machines, and that's what "determined your race from your voice" is pretty close to.
    14. xwolfi ◴[22 Oct 25 03:56 UTC] No.45664749{3}[source]▶
    All my Indian colleagues say "I agree with the same", this "the same" turn of phrase was so strange to me I had to ask (I'm French, so I have my own silly quirks, like I forget non-vocal plural(s<-- see, often I don't write that s)). They told me it was like that in Hindi so they just reproduce the pattern and it's grammatically acceptable.

    For French people like me, false friends are immediately noticeable: for instance, "actually" to mean "now" instead of "in fact".

    replies(1): >>45677405 #
    15. vessenes ◴[22 Oct 25 04:00 UTC] No.45664768[source]▶
    Pre-nerf the 4o voice model had a wide range of expressivity, and it would match affect (still tries to do this) and idiolect of listeners if asked. Nowadays there's a list of accents that are considered "hate-ish" and a list that aren't.

    I will elide the rant inside me that west coast 20 somethings get to decide if speaking in a certain accent is racist or "bad". But it's a heartfelt rant.

    16. Xmd5a ◴[23 Oct 25 02:07 UTC] No.45677405{4}[source]▶
    >like I forget non-vocal plural(s<-- see, often I don't write that s)).

    c'est infernal. infernal