←back to thread

269 points mtlynch | 1 comments | | HN request time: 0s | source
Show context
sgarland ◴[] No.43955920[source]
> Ship first, worry about scale later

This is repeated constantly, but I fear that it is internalized as “write shitty code and throw money at it later.” If you have taken the time to learn your language well, you can avoid a lot of really bad decisions that don’t cost you additional time.

Similarly, on the infra side of things (where this advice is usually doled out), maybe take the time to have a modicum of understanding about the tools you’re building on. If you’re using a DBaaS, your vendor almost certainly has monitoring built-in, often for free, or a nominal cost. USE IT, and learn what it is you’re looking at. “The DB is slow” could be anything from excessive row locks due to improperly-held transactions to actually hitting an underlying resource limit – and for the latter, 9/10 it’s a symptom of something that’s misconfigured, or not understanding your RDBMS’ operation.

For example, do you have a write-heavy table with a UUIDv4 PK, lots of columns that are heavily indexed, and some medium-large JSON blobs in it? Congratulations, you’ve created Postgres’ (and MySQL, but for different reasons) worst nightmare. Every write is amplified by the indexes, and even if you’re doing an UPDATE and are only hitting one of the indexed columns, all of them will be rewritten. The UUIDv4 PK means your WAL traffic is going to skyrocket from all the full page writes, and if your JSON blobs are big enough to be unwieldy, but not big enough to have be TOASTed, that’s another huge amplification to writes. All of this can easily result in hitting IOPS limits, network bandwidth limits, or CPU saturation from additional queries piling up while this one is dealt with, and all of it could be easily avoided by having a basic understanding of your tooling.

replies(2): >>43956126 #>>43956188 #
chrisweekly ◴[] No.43956126[source]
This is an awesome comment. As someone who doesn't spend much time in the DB tier, I especially appreciate the real-world details in your final paragraph. Do you have a blog where you write about this stuff, and/or have any particular recommendations for learning or reference materials? I'm sure there's no substitute for experience when it comes to developing an intuition for good db design, but I'd be grateful for your expert advice if you have more to share. TIA
replies(1): >>43956215 #
sgarland ◴[] No.43956215[source]
I don’t generally blog about it, but I may at some point.

For specific discussions on what I wrote about, I recommend (other than reading Postgres’ docs, of course) this [0] and this [1], and anything else on those sites.

If you like technical podcasts, postgres.fm [2] is pretty good.

[0]: https://www.cybertec-postgresql.com/en/hot-updates-in-postgr...

[1]: https://www.enterprisedb.com/blog/impact-full-page-writes?la...

[2]: https://postgres.fm/

replies(1): >>43958296 #
1. chrisweekly ◴[] No.43958296[source]
Awesome! Thank you!

Also, you really should consider creating a blog -- your writing is good, and well-informed.