Most active commenters
  • hnick(4)

←back to thread

3883 points kuroguro | 26 comments | | HN request time: 0.002s | source | bottom
Show context
breakingcups ◴[] No.26296724[source]
It is absolutely unbelievable (and unforgivable) that a cash cow such as GTA V has a problem like this present for over 6 years and it turns out to be something so absolutely simple.

I do not agree with the sibling comment saying that this problem only looks simple and that we are missing context.

This online gamemode alone made $1 billion in 2017 alone.

Tweaking two functions to go from a load time of 6 minutes to less than two minutes is something any developer worth their salt should be able to do in a codebase like this equipped with a good profiler.

Instead, someone with no source code managed to do this to an obfuscated executable loaded with anti-cheat measures.

The fact that this problem is caused by Rockstar's excessive microtransaction policy (the 10MB of JSON causing this bottleneck are all available microtransaction items) is the cherry on top.

(And yes, I might also still be salty because their parent company unjustly DMCA'd re3 (https://github.com/GTAmodding/re3), the reverse engineered version of GTA III and Vice City. A twenty-year-old game. Which wasn't even playable without purchasing the original game.)

replies(40): >>26296812 #>>26296886 #>>26296970 #>>26297010 #>>26297087 #>>26297123 #>>26297141 #>>26297144 #>>26297184 #>>26297206 #>>26297323 #>>26297332 #>>26297379 #>>26297401 #>>26297448 #>>26297480 #>>26297806 #>>26297961 #>>26298056 #>>26298135 #>>26298179 #>>26298213 #>>26298234 #>>26298624 #>>26298682 #>>26298777 #>>26298860 #>>26298970 #>>26299369 #>>26299512 #>>26299520 #>>26300002 #>>26300046 #>>26301169 #>>26301475 #>>26301649 #>>26301961 #>>26304727 #>>26305016 #>>26311396 #
masklinn ◴[] No.26296886[source]
> The fact that this problem is caused by Rockstar's excessive microtransaction policy (the 10MB of JSON causing this bottleneck are all available microtransaction items) is the cherry on top.

For what it's worth, 10MB of JSON is not much. Duplicating the example entry from the article 63000 times (replacing `key` by a uuid4 for unicity) yields 11.5MB JSON.

Deserialising that JSON then inserting each entry in a dict (indexed by key) takes 450ms in Python.

But as Bruce Dawson oft notes, quadratic behaviour is the sweet spot because it's "fast enough to go into production, and slow enough to fall over once it gets there". Here odds are there were only dozens or hundreds of items during dev so nobody noticed it would become slow as balls beyond a few thousand items.

Plus load times are usually the one thing you start ignoring early on, just start the session, go take a coffee or a piss, and by the time you're back it's loaded. Especially after QA has notified of slow load times half a dozen times, the devs (with fast machines and possibly smaller development dataset) go "works fine", and QA just gives up.

replies(11): >>26297203 #>>26297314 #>>26298126 #>>26298269 #>>26298511 #>>26298524 #>>26300274 #>>26301081 #>>26302098 #>>26305727 #>>26306126 #
1. hnick ◴[] No.26298126[source]
> Plus load times are usually the one thing you start ignoring early on, just start the session, go take a coffee or a piss, and by the time you're back it's loaded.

In GTA V, when I tried to enjoy multiplayer with my friends the abysmal load times were what killed it for me.

You actually have to load into the game world - which takes forever - before having a friend invite you to their multiplayer world - which takes forever, again.

So both a coffee, and a piss. Maybe they fixed that now?

replies(5): >>26298242 #>>26298478 #>>26298536 #>>26298577 #>>26298673 #
2. simias ◴[] No.26298242[source]
I agree. I played GTA online for a bit and quite enjoyed it but I haven't touched it in a while and the insane loading times are a big reason why.

It kind of baffles me that they haven't bothered to fix this trivial issue when the result is to cut 4 entire minutes of loading time.

replies(1): >>26300141 #
3. agret ◴[] No.26298478[source]
Then when you want to actually do an activity like a deathmatch you have to wait for matchmaking and then the loading - takes forever. Once you are finally in a match it's okay but as soon as the match ends you have to wait for the world to load again and then queue again which takes bloody forever. Spend 2hrs playing the game and have only a few matches, more time spent looking at loading screens than actually playing anything.
replies(2): >>26300300 #>>26301380 #
4. zmix ◴[] No.26298536[source]
> So both a coffee, and a piss.

Reminds me on loading "G.I. Joe" (from Epyx) on the C64 with a 1541 floppy disk. However, the long loads came after every time you died and meant you also had to swap 4 disks.

replies(1): >>26298585 #
5. ajacksified ◴[] No.26298577[source]
They didn't fix it. I tried a few days ago, because it's a really fun game... except for these seemingly easy to fix issues that are huge barriers.
6. hnick ◴[] No.26298585[source]
I remember as a kid I went to someone's birthday party in the 80s and we wanted to play a karate themed game on something that used a cassette tape. It took so long to load we went and played outside!

To be fair to GTA V, I don't think my installation was on a SSD because it was 50GB or something at the time (now it's 95GB?), but that said when it released SSDs were not as cheap or widespread as they are now so that's their problem. The linked article shows the initial load to be much shorter which did not match my experience.

replies(4): >>26300021 #>>26301730 #>>26302175 #>>26303483 #
7. thaumasiotes ◴[] No.26298673[source]
> You actually have to load into the game world - which takes forever - before having a friend invite you to their multiplayer world - which takes forever, again.

Is that... the same problem? Is microtransaction data different in your friend's multiplayer world than it is in the normal online world?

replies(1): >>26298870 #
8. hnick ◴[] No.26298870[source]
The article mentions story mode loading as well as online loading, but as I mentioned in another comment the story time shown there is much lower than what I experienced, probably because SSDs are now standard and were rarer in 2013 (I could not justify 50GB+ on this one game at the time). So it may be a mixture of factors.
9. jcims ◴[] No.26300021{3}[source]
Not to one up but we didn’t have any storage for our c64 for the first year or so. We would team up to enter a game from Byte or whatever (one reader, one typer, one proofreader) and then protect that thing with our lives all weekend to keep people from unplugging it. The machine code games were the easiest to type but if they didn’t work you were kind of hosed lol.
replies(1): >>26334676 #
10. jjoonathan ◴[] No.26300141[source]
Back in dialup/DSL days I discovered a texture compression issue in America's Army (the free US Army game) that doubled its download/install size. Typical download times were about a day and the resume mechanism was poor, so this had the potential to save a lot of grief, not to mention hosting costs. I emailed them, they banned me for hacking, and the next version still had the same issue. Shrug.
replies(2): >>26301028 #>>26344626 #
11. ZuLuuuuuu ◴[] No.26300300[source]
Judging from your word choice "deathmatch" and your experience with long loading/matchmaking times I guess you might be a fellow Quake Champions player. Even if you are not, I agree that long loading times are a mood killer when you just want to play a couple of quick matches after work in your limited free time. It is even worse when you know the game's development is abandoned and it will never get fixed, even though you enjoy the game itself.
replies(1): >>26300591 #
12. franga2000 ◴[] No.26300591{3}[source]
GTA V has a deathmatch mode and the parent comment sounds like it's talking about that. Especially the "once it's over, you need to load into the primary session, then wait for matchmaking, then wait for loading again" sounds exactly like GTA V.
replies(1): >>26300661 #
13. ZuLuuuuuu ◴[] No.26300661{4}[source]
Ah I see, thanks for the clarification.
14. jack_riminton ◴[] No.26301028{3}[source]
That's hilarious. Out of interest who in the company did you email?
replies(1): >>26302363 #
15. yholio ◴[] No.26301380[source]
> more time spent looking at loading screens than actually playing anything.

This could easily compete for the most expensive bug in history, up there with the Pentium Bug. It might have halved the revenue potential of a billion dollar franchise.

16. mst ◴[] No.26301730{3}[source]
Oh gods, now I'm reminded of a cassette based game where there was one enemy where if he got a solid hit on you, you got bumped back to the beginning of the level.

Which meant the game displayed "rewind to mark 500 and then hit play" and you had to do that to restart lolsob.

replies(1): >>26322973 #
17. vagrantJin ◴[] No.26302175{3}[source]
That we can say the install bundle is 50 gigs with a straight face though? I remember not long ago games that were 8 gigs caused mayhem.

I suppose devs dont care about that as long as their QA allows it.

replies(1): >>26309481 #
18. jjoonathan ◴[] No.26302363{4}[source]
I was only able to find contact information for one person who I knew was probably technical (from their posts), so I sent it to them.

I never learned the "other side" of this story, but a few years later the same dev team tried to recruit me at a CS contest, to which I politely declined.

More details: I was young, without credit card, and gaming on a mac. AA was free and mac compatible. For a while -- apparently mac ports of unreal engine games were approximately all done by a single very productive contractor and from what I understand the US Army, uhh, stopped paying him at some point. So he stopped releasing the mac ports. From my point of view, this meant that I could only play with other mac users and couldn't use any of the fancy new maps. Logs indicated that the compatibility problems with the new maps were not particularly deep, so I got to parsing the unreal map files and was able to restore compatibility by deleting the offending objects. I implemented texture decoding/encoding mostly for curiosity and because textures were well documented in the reverse engineering "literature." I imagined a workflow where someone would bulk export and re-import textures and aside from the texture names the one piece of metadata I needed was the format: RGBA (uncompressed) or DXT (compressed)? I realized that I could easily identify DXT compression from the image histogram, so I didn't need to store separate metadata. Nifty! But it didn't work. Lots of textures stored in uncompressed RGBA8888 "erroneously" round-tripped to DXT. After poring over my own code, I eventually realized that this was because on many of the textures someone had enabled DXT compression and then disabled it, dropping the texture quality to that of DXT while bloating the texture size to that of RGBA8888 (other textures were still stored as DXT, so compression itself was still working). I wrote a quick tool to add up the wasted space from storing DXT compressed textures in uncompressed RGB format and it came out to about half the total disk space, both before and after the top level installer's lossless compression. They could have re-enabled compression on most of the textures where they had disabled it without loss in quality, and if they had wanted a list of such textures I would have been able to provide it, but it didn't go down that way. When I shared what happened with my father, who had served, his reaction was "Now that's the Army I know!"

replies(1): >>26313818 #
19. rasz ◴[] No.26303483{3}[source]
C64 tape with turbo was actually faster (~500 Bytes/s) than non turbo Floppy (~400 Bytes/s).

Many 8bit Atari owners will have horror memories of aborted Tape loads. After over 20 years someone finally discovered a bug in original ATARI Tape loading ROM routine resulting in randomly corrupted loads, no amount of sitting motionless while the game loads would help in that case :)

20. hnick ◴[] No.26309481{4}[source]
I read that after patches GTA V is now around 95GB.

Call of Duty: Black Ops Cold War is around 200GB fully installed with all features (HD textures etc).

Some people have insinuated this is intentional to crowd out other games on console hard disks and make moving away from CoD have an opportunity cost. It's probably just laziness.

I haven't looked into it in the past but some prior offenders had a lot of space wasted from uncompressed multi-lingual audio. Instead of letting us choose a language it installs them all so you can switch in game, and uncompressed for saving the CPU for game logic. For CoD the optional HD texture pack is 38GB so that's still a lot unaccounted for.

replies(2): >>26315204 #>>26345841 #
21. jack_riminton ◴[] No.26313818{5}[source]
huh, I wonder if the technical person send it to management for a decision?
22. STRML ◴[] No.26315204{5}[source]
Titanfall did this. If I recall correctly, the full install size was about 48GB, 35GB of which was just uncompressed audio. And that was back in the days when 120GB (or less) SSDs were common. A total self-own and never fixed or understood.

It's not like decoding audio takes enough time on any modern multi-core processor to disrupt the game loop. It's not even on the radar.

23. da_chicken ◴[] No.26322973{4}[source]
Maybe this is what they really meant by "GOTO considered harmful".
24. zmix ◴[] No.26334676{4}[source]
Oh, wow! This was some real hardcore sh*t! :-)
25. wing-_-nuts ◴[] No.26344626{3}[source]
Oh man, this brings back memories. That game was a great tactical shooter back in the day. Sadly my PC was unable to keep up with the higher requirements required by it's updates.
26. vagrantJin ◴[] No.26345841{5}[source]
I mean. Jesus christ. Devs used to be proud they could optimize and run their programs on slow machines. What happened? Thats like making a game for casual players but only < 5% of the global population can even afford to play it. How does this make business sense?? I'd understand if it was a game breaking new ground with some hectic VR or something like that but it isnt.