Most active commenters

    ←back to thread

    The Walt Disney Company and OpenAI Partner on Sora

    (openai.com)
    236 points inesranzo | 17 comments | 11 Dec 25 14:05 UTC | HN request time: 0.001s | source | bottom
    Show context
    giancarlostoro ◴[11 Dec 25 14:40 UTC] No.46231943[source]▶
    This will not end well for Disney, there were certain historical characters removed from Sora 2 because people kept making racist videos that are hard to censor, and it became increasingly unhinged. This feels like another circular investment where Disney is hoping to make money back I'm sure. On the other hand, assuming they do the freemium stuff, I look forward to making a few videos of my daughters favorite Disney princesses "talking" to her.
    replies(21): >>46231963 #>>46232027 #>>46232049 #>>46232134 #>>46232217 #>>46232316 #>>46232450 #>>46232484 #>>46232594 #>>46232634 #>>46232732 #>>46232770 #>>46232998 #>>46233138 #>>46233293 #>>46233477 #>>46234040 #>>46234843 #>>46235615 #>>46235942 #>>46236727 #
    impulser_ ◴[11 Dec 25 15:01 UTC] No.46232217[source]▶
    I guess it will depend on how good their security is, because I'm assuming Disney is entering this deal with hard guidelines on what will be allowed.
    replies(2): >>46232401 #>>46232756 #
    1. bilbo0s ◴[11 Dec 25 15:16 UTC] No.46232401{3}[source]▶
    This.

    Don't believe for a second that Sora will allow you to make racist content with Disney characters.

    That said, there are a lot of other models out there that care about neither licensing nor alignment. So those will allow you to make racist content. Then you can do whatever you like with that generated content.

    A lot of IP owners will learn that there is more than one way to skin a cat. It's easier than people think to turn children's characters, like say, Hermoine, into a raging racist. And there's very little technically speaking that they can do to stop it.

    But yes, on OpenAI specific properties, they can definitely stop it dead in its tracks. They can even get better at stopping it over time. In fact, the more users try to generate it, the better the system will get at stopping it.

    replies(11): >>46232460 #>>46232464 #>>46232466 #>>46232469 #>>46232486 #>>46232641 #>>46232648 #>>46233618 #>>46233677 #>>46233712 #>>46233869 #
    2. conartist6 ◴[11 Dec 25 15:20 UTC] No.46232460[source]▶
    Once you make the content it's just content, no? How the hell are you going to ban racism? A lot of racism is "dog whistle" stuff anyway -- it's designed to convey a message to people who already know what is being suggested while seeming innocuous to any enforceable standards of decency... The racists will surely have no trouble bending the models to produce Disney content that is in practice used to promote and celebrate racism.
    3. zrobotics ◴[11 Dec 25 15:20 UTC] No.46232464[source]▶
    I'm sure that's what Disney's lawyers specified in the contracts and what their execs expect. However, judging by how LLM controls have gone in the past, I'm fully expecting to see a slew of awful content featuring Disney's characters in the days after this launches. OpenAI also probably won't ever be able to actually stop people from generating harmful content with the characters, but the volume of awful stuff will probably eventually slow down as people get bored and move onto the next controversial thing.
    4. dagmx ◴[11 Dec 25 15:20 UTC] No.46232466[source]▶
    How can you prevent it from making that content? People will find a way to combine certain words together to make it happen.
    5. simonw ◴[11 Dec 25 15:20 UTC] No.46232469[source]▶
    Given the creativity of the jailbreaking community I will be very impressed if OpenAI manage to reliably prevent Sora from creating disagreeable content with Disney characters.
    replies(2): >>46232722 #>>46232731 #
    6. chaostheory ◴[11 Dec 25 15:21 UTC] No.46232486[source]▶
    Based on what I’ve seen on Sora, it’s likely possible but only for a day or a few hours. They’ve been getting better and faster at censoring.

    I agree. Those characters are likely safe on Sora

    replies(1): >>46233144 #
    7. PunchyHamster ◴[11 Dec 25 15:33 UTC] No.46232641[source]▶
    > Don't believe for a second that Sora will allow you to make racist content with Disney characters.

    Yes, because AI's so far have been oh so resilient to jailbreaks and oh so great at picking out the potentially "not aligned with corporate values" content...

    8. lm28469 ◴[11 Dec 25 15:34 UTC] No.46232648[source]▶
    > Don't believe for a second that Sora will allow you to make racist content with Disney characters.

    It's not racist, it's an historically accurate depiction of 1930s Germany under the authority of a significant leader who may or may not be controversial today

    9. ◴[11 Dec 25 15:39 UTC] No.46232722[source]▶
    10. sarchertech ◴[11 Dec 25 15:39 UTC] No.46232731[source]▶
    If they’re jail breaking it, they should already be able to get it to create Disney characters without this deal.
    replies(1): >>46233123 #
    11. andrew_lettuce ◴[11 Dec 25 16:06 UTC] No.46233123{3}[source]▶
    Almost like this is not about the value of the deal, but the payoff to both sides from the headlines?
    12. andrew_lettuce ◴[11 Dec 25 16:08 UTC] No.46233144[source]▶
    That's still whack-a-mole though, just because they're rolling out new virus definitions quicker doesn't do anything to prevent zero day exploits
    13. corobo ◴[11 Dec 25 16:39 UTC] No.46233618[source]▶
    How are they going to stop it?

    A certain combination of nonstandard characters will make an AI character drop an n-word no problem

    I guess they could chuck the output through whisper or something to see if it transcribes back to anything dodgy?

    LLM security feels very ball of sand held together with duct tape haha

    14. dragonwriter ◴[11 Dec 25 16:43 UTC] No.46233677[source]▶
    > Don't believe for a second that Sora will allow you to make racist content with Disney characters.

    Don’t believe for a minute that whatever filters it uses will be sensitive enough to the way racist content is constructed to stop people from doing just that.

    replies(1): >>46235408 #
    15. butlike ◴[11 Dec 25 16:46 UTC] No.46233712[source]▶
    Great, so censorship by attrition.
    16. doug_durham ◴[11 Dec 25 16:57 UTC] No.46233869[source]▶
    I don't think that you will ever be able to generate Disney characters with the Sora app. Sora is both a model and an app. Instead I think that there will be a heavily guardrailed specialized app where you can do some highly restricted things for the opportunity for the content to show up on Disney+. Think of it as "Disney art! Powered by Sora".
    17. mrguyorama ◴[11 Dec 25 18:51 UTC] No.46235408[source]▶
    "Don't let the user do racist shit" is literally AGI Hard territory of problem solving.

    It's right up there with "Let kids communicate anonymously but not to perverts" and "Is this porn or educational?"