The Grug Brained Developer (2022)

“Good debugger worth weight in shiny rocks, in fact also more”

I’ve spent time at small startups and on “elite” big tech teams, and I’m usually the only one on my team using a debugger. Almost everyone in the real world (at least in web tech) seems to do print statement debugging. I have tried and failed to get others interested in using my workflow.

I generally agree that it’s the best way to start understanding a system. Breaking on an interesting line of code during a test run and studying the call stack that got me there is infinitely easier than trying to run the code forwards in my head.

Young grugs: learning this skill is a minor superpower. Take the time to get it working on your codebase, if you can.

I'd love to use a real debugger but as someone who has only ever worked at large companies, this was just never an option. In a microservices mesh architecture, you can't really run anything locally at all, and the test environment is often not configured to allow hooking up a stepping debugger. Print debugging is all you have. If there's a problem with the logging system itself or something that crashes the program before the logs can flush, then not even that.

At my company our system is composed of 2 dozen different services and all of them can run locally in minikube and easily be debugged in jetbrains.

Where do configs and secrets come from? Also, big company = hundreds of microservices.

In my experience you just slap minikube or k3s on your dev machine, and treat it as any other environment. Argo, helm, kustomize, whatever can all work against a local single-node cluster just fine. It takes some effort to make sure your configs are overridable per-environment, but it’s worth doing. (And something you’re hopefully doing anyway if you’re doing any kind of integration/test environment.)

It also requires that each of your services can scale down as well as they can scale up… none of them should be so huge that your whole cluster can’t fit on a single machine, if you’re just simulating one “request” at a time. (Single instances of everything, don’t reserve memory, etc.) There’s use cases where this isn’t practical, but in most cases it’s very doable.