Most active commenters

    ←back to thread

    My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

    (simonwillison.net)
    577 points simonw | 12 comments | 29 Jul 25 13:45 UTC | HN request time: 0.001s | source | bottom
    Show context
    croes ◴[29 Jul 25 13:59 UTC] No.44723485[source]▶
    I bet the training data included enough space invader cloned in JS
    replies(3): >>44723515 #>>44723527 #>>44724105 #
    1. jplrssn ◴[29 Jul 25 14:02 UTC] No.44723515[source]▶
    I also wouldn't be surprised if labs were starting to mix in a few pelican SVGs into their training data.
    replies(3): >>44723675 #>>44723681 #>>44723878 #
    2. quantumHazer ◴[29 Jul 25 14:16 UTC] No.44723675[source]▶
    SVG benchmarking is a thing since GPT-4, so probably all major labs are overfitting on some dataset ov svg images for sure
    3. diggan ◴[29 Jul 25 14:16 UTC] No.44723681[source]▶
    Even "accidentally" it makes sense that "SVGs of pelicans riding bikes" are now included into datasets used for training as it has spread as a wildfire on the internet, making it less useful as a simple benchmark.

    This is why I keep all my benchmarks private and don't share anything about them publicly, as soon as you write about them anywhere publicly they'll stop being useful in some months.

    replies(1): >>44723789 #
    4. toyg ◴[29 Jul 25 14:24 UTC] No.44723789[source]▶
    > This is why I keep all my benchmarks private

    This is also why, if I were an artist or anyone commercially relying on creative output of any kind, I wouldn't be posting anything on the internet anymore, ever. The minute you make anything public, the engines will clone it to death and turn it into a commodity.

    replies(2): >>44724505 #>>44724610 #
    5. simonw ◴[29 Jul 25 14:30 UTC] No.44723878[source]▶
    I'll believe they are doing that when one of the models draws me an SVG that actually looks like a pelican.
    replies(1): >>44724540 #
    6. __mharrison__ ◴[29 Jul 25 15:21 UTC] No.44724505{3}[source]▶
    Somewhat defeats the purpose of being an artist, doesn't it?
    replies(3): >>44724849 #>>44725086 #>>44734800 #
    7. __mharrison__ ◴[29 Jul 25 15:23 UTC] No.44724540[source]▶
    Someone needs to craft a beautifully bike donned by a pelican, throw in some seo, and see how long it takes a model to replicate it.

    Simon probably wouldn't be happy about killing his multi-year evaluation metric though...

    replies(1): >>44724587 #
    8. simonw ◴[29 Jul 25 15:27 UTC] No.44724587{3}[source]▶
    I would be delighted.

    My pelican on a bicycle benchmark is a long con. The goal is to finally get a good SVG of a pelican riding a bicycle, and if I can trick AI labs into investing significant effort in cheating on my benchmark then fine, that gets me my pelican!

    9. debugnik ◴[29 Jul 25 15:29 UTC] No.44724610{3}[source]▶
    That makes it so much harder to show art to people and market yourself though.

    I considered experimenting with web DRM for art sites/portfolios, on the assumption that scrappers won't bother with the analog loophole (and dedicated art-style cloners would hopefully be disappointed by the quality), but gave up because of limited compatible devices for the strongest DRM levels, and HDCP being broken on those levels anyway. If the DRM technique caught on it would take attackers, at most, a few bucks and hours once to bypass it, and I don't think users would truly understand that upfront.

    10. toyg ◴[29 Jul 25 15:48 UTC] No.44724849{4}[source]▶
    Defeating the purpose of creating almost anything, really.

    AI is definitely breaking the whole "labor for money" architecture of our world.

    11. zhengyi13 ◴[29 Jul 25 16:07 UTC] No.44725086{4}[source]▶
    Eeeehhhh.

    Maybe the thing to do is provide public, physical exhibits of your art in search of patronage.

    12. diggan ◴[30 Jul 25 14:36 UTC] No.44734800{4}[source]▶
    Worth noting, in case people weren't aware, but bunch of people have different motivations for being artists, not everyone wants everything they do to be as widely shared as possible, some are happy playing/drawing/whatever for a small group of people, and not even making money off it.