←back to thread

The Data Engineering Handbook

(github.com)
183 points matthewhefferon | 2 comments | 19 Nov 24 17:31 UTC | HN request time: 0s | source
Show context
mettamage ◴[20 Nov 24 04:29 UTC] No.42190831[source]▶
Is there a data analyst handbook? Starting a role in that soon (switching from SWE - tired of full-time programming)
replies(1): >>42191266 #
blub ◴[20 Nov 24 06:24 UTC] No.42191266[source]▶
Is this still a career path worth investing in? It seemed super hot a couple of years ago and then people stopped talking about it. Saw many comments complaining about lack of jobs.
replies(3): >>42191512 #>>42191534 #>>42193851 #
fifilura ◴[20 Nov 24 07:26 UTC] No.42191534[source]▶
I think it should be worth it. For the reason that you are closer to the product you are building, you help defining what to build.

As opposed to programming which is more like plumbing work.

replies(1): >>42192652 #
1. gregw2 ◴[20 Nov 24 10:47 UTC] No.42192652[source]▶
Believe me, there's a lot of plumbing moving stuff from point A to B and dealing with poop ("dirty data" is the industry euphemism) in the data engineering and data analyst space.

In my more analytic moments I try to convince myself that data engineering and analysis is like chemical refining, creating useful byproducts out of raw liquids, but in my cynical moments, the plumbing metaphors for it are just so much more evocative.

replies(1): >>42193905 #
2. fifilura ◴[20 Nov 24 13:48 UTC] No.42193905[source]▶
Still, somehow, I think it is where it all started in the 1950s. "I have all these numbers, I need a machine to help me do something useful".

And then after this came a huge industry with programmers.

For me it is more like back-to-basics.