How I cut GTA Online loading times by 70%

Any sufficiently large institution, over time, will prioritise self-preservation over achieving their core mission. This is sociology 101. Once a company has enough users to make it hard or impossible to measure immediate performance, self-preservation is achieved with internal manoeuvering and selling to execs.

In games the added reason to be slow is that game code is by definition some of the least mission critical code one could find (competes with 90% of the internet web code). Your Linux or Windows code might run a hospital's infrastructure or a rover on another planet. A game on the other hand can launch with bugs the size of your windshield, and can stay like that forever as long as people still pay. And people will pay because games are not unlike a drug for many people.

As such most game coding teams and coders are "trained" to cut every corner and skimp on every precaution. They're not needed beyond a very low baseline as far as software is concerned.

Look at the amount of bugs or cheats incredibly popular games like GTA or CoD have. These are billion dollar a year franchises that leave all this crap on the table despite all the money they make. They have all the resources needed, it's a conscious call to proceed like this, to hire teams that will never be qualified enough to deliver a high quality product and will be encouraged to cut corners on top of that.

Source: a long time ago I worked for a major game developer in a senior management role (unrelated to the dev activity) and left after feeling like "facepalm" for too long in every single SM meeting.

The HN discussion[1] was started by an article that provided numbers that seemed to suggest that Wikipedia's spending was slowly spiraling out of control.

1: https://news.ycombinator.com/item?id=14287235

After tangentially working on this for a long time I'd say that the core issue is so deeply ingrained in human psyche that it may not even be a matter of education (starts early), let alone organization (starts happening when everything else is "set in stone"). There's no organizational structure that fits the types of activities we humans tend to do these days and that can deliver low latency, consistent results at scale. We mitigate issues in one area by accentuating them in others.

You can have one large flat structure but the load on the single coordinating circuit (the manager) will compromise quality. You can split the thing in multiple, individually coordinated units but the added layer of coordination and latency will compromise performance.

Maybe some form of general purpose but not quite full AI, something that combines human like intelligence and engineering like consistency, might be able to do what humans are supposed to but without the variability of humans (which is both good and bad).

This doesn't happen because it would reduce the power of top decision makers and potentially impact profits. e.g. a customer might ask for a chronologically ordered timeline on Facebook, but that would harsh engagement metrics, revenue, etc. If stuff like this did happen more often though, you'd get products and services that more often achieve their stated aims.

We don't really know what it is simply because we haven't introduced any "real" AI anywhere, let alone:

>> some form of general purpose but not quite full AI

Talking about efficiency as you scale up large organizations, it's inevitable that humans will introduce delays and variability in the work which cannot be eliminated because it's human nature, biologically and psychologically. Since humans can't change on a timescale that makes this discussion relevant, the only way for very large organizations (like large companies or governments) to operate just as efficiently as small ones is if they rely (quasi)exclusively on some AI that's as capable as top individuals at delivering results but with fewer of the drawbacks. Not only would it not operate as 100.000 distinct entities but as one or a few, it would also run consistently and predictably.

https://randomascii.wordpress.com/2021/02/16/arranging-invis...

I've seen plenty of interesting bugs, best I found personally was a compiler that was outputting files 1 byte at a time.

Games are riff with these sorts of bugs, but the volume of released games vs other types of software makes any sort of comparison unfair.

No comparison is entirely fair but I think your objection is unfounded. The quantity of games being released is irrelevant when we're talking about the quality of their code. Which is really bad for most big titles.

If anything the comparison was unfair in the other direction. Sure, an OS or browser have a lot of bugs. But if games and their code had a fraction of the scrutiny something like Windows gets you might just find out that 4 lines of code were written "by the book". It's something any honest game dev will confirm, game code is a stack of spaghetti code on top of more spaghetti code. The philosophy is that you can just go ahead with bad code because you can always fix it with a patch later on. Then you notice there's no widespread pushback from gamers (because unless it's an absolute bomb there won't be) and move on with the next round of "features", nobody has time for fixing bugs or combing the spaghetti.

One other problem is that eventually some people coming from the gaming industry will end up switching to other types of software development but will stick to the philosophy. I've done a lot of hiring over my career and of one thing I'm certain. Whenever someone came with most of their career in game development or most of the recent experience I asked for the CV to be put at the bottom of the stack. It was a lesson I learned the hard way.

Issues like the low criticality of game code, the "crunch" work style, the idea that you should just get it out there as quickly as possible, the lack of serious scrutiny into the matter, etc. all compound each other to create a coding (and coordination) style that's hard to shake off.

I should have clarified.

Doing an absolute numerical comparison in terms if "bad game code" to "bad B2C code" is unfair, because now days there is a lot more game dev code than B2C code (if we ignore the web, which is a rather lot of code to ignore I'll admit :) ).

All that said, most of the games on my phone run very well. Most of the games on my PC run very well. Do the massive over budget AAA titles have issues? Of course. Human organizations are really bad at creating large projects, we all know that social interaction overhead scales non-linearly with # of people, large projects get bogged down.

(Now throw in B2B software where the CTO makes the purchasing decisions and only people lower level in the company have to use it. All of a sudden enterprise software starts to suck as much as AAA games!)

> The philosophy is that you can just go ahead with bad code because you can always fix it with a patch later on.

Sure, for AAA games. But plenty of games that I play don't have those sorts of issues.

Meanwhile my Windows start menu only opens every other time I press the start menu button. This happened for years, then it was fixed for a bit, then a few patches later it started happening again.

The start menu on Windows should be the 2nd to last thing that breaks, right behind the mouse driver.

Without huge testing efforts, large software projects will collapse under its own weight. The type of software isn't material to the problem.

> One other problem is that eventually some people coming from the gaming industry will end up switching to other types of software development but will stick to the philosophy.

I've hired plenty of ex-game devs, I agree they have some holes in their skills. But I've also found out that if you sit them down they can write code that traditional software engineers couldn't manage. Spending too long in any one paradigm fixes ones mindset into that paradigm.

I also agree that the quality of code someone puts out is highly influenced by the environment they work in. But I've worked with plenty of ex game devs who take pride in writing correct code the first time around.

> Whenever someone came with most of their career in game development or most of the recent experience I asked for the CV to be put at the bottom of the stack.

That's a shame, because I wouldn't have accomplished some of the things I've done in my career if I hadn't hired a few game developers along the way! I've seen some stupid crazy stuff get pulled off, code that "shouldn't be able to exist", but did exist, and was rock solid reliable, all written by game developers.

People are people, and sure 9 out of 10 the person who bangs on kernels for a living is probably super careful about every line of code they write, but I'm not going to judge an entire field because the most bureaucratic outputs of that field are horrible.

That'd be like saying everyone who writes web apps is an incompetent developer because Jira is slow.

Or that all software developers are bad because, honestly, the majority of users only notice what we make when it breaks.

Well, we only hear about the small % of games that have huge problems, not the larger number of games that work just fine out of the box.

Normally the response from management was "but will that increase our profits?", to which my answer was "eventually, yes"

What really stuck out is that Buffet said its the most important thing about Business and he didn't learn it in Business school. Same! I did an MBA and it was the biggest elephant in the room.

"if you’re not watchful, the process can become the thing. This can happen very easily in large organizations. The process becomes the proxy for the result you want. You stop looking at outcomes and just make sure you’re doing the process right"