←back to thread

149 points rbanffy | 2 comments | | HN request time: 0s | source
Show context
vundercind ◴[] No.42151467[source]
Probably 80% of notable performance problems I’ve seen in the kinds of systems that things like Django and Ruby get used for have been terrible queries or patterns of use for databases (I’ve seen 1,000x or worse costs for this versus something more-correct) and nearly all of the other 20% has been areas that plainly just needed some pretty straightforward caching.

The nice thing about that is that spotting those, and the basic approach to fixing them, if not the exact implementation details, are cross-platform skills that apply basically anywhere.

I actually can’t recall any other notable performance problems in those sorts of systems, over the years. Those are so common and the fixes so effective I guess the rest has just never rated attention. I’ve seen different problems in long-lived worker processes though (“make it streaming—everything becomes streaming when scale gets big enough” is the usual platform-agnostic magic bullet in those cases)

A bunch of TFA is basically about those things, so I’m not correcting it, more like nodding along.

Oh wait I just thought of another I’ve seen: serving large files through a scripting language, as in, reading it in and writing it back out with a scripting language. You run into trouble at even modest scale. There’s a magic response header for that, make Nginx or Apache or whatever serve it for you, it’s a fix that’s typically deleting a bunch of code and replacing it with one or two lines. Or else just use s3 and maybe signed URLs like the rest of the world. Problem solved.

replies(4): >>42151984 #>>42152017 #>>42154568 #>>42155730 #
megaman821 ◴[] No.42151984[source]
I have had to combine files into a zipped file on demand before. It is hard to avoid the inherent slowness of that.
replies(3): >>42152323 #>>42153663 #>>42155601 #
1. ch4s3 ◴[] No.42152323[source]
Interesting, was there a business reason to not do that in the background somewhere?
replies(1): >>42153403 #
2. megaman821 ◴[] No.42153403[source]
Yeah, very non-technical users that won't check their email or click on a notification when the zip file is ready for them.