Most active commenters
  • mort96(4)
  • writebetterc(3)

←back to thread

The Grug Brained Developer (2022)

(grugbrain.dev)
1087 points smartmic | 17 comments | 17 Jun 25 20:24 UTC | HN request time: 1.498s | source | bottom
Show context
titanomachy ◴[17 Jun 25 23:45 UTC] No.44305194[source]▶
“Good debugger worth weight in shiny rocks, in fact also more”

I’ve spent time at small startups and on “elite” big tech teams, and I’m usually the only one on my team using a debugger. Almost everyone in the real world (at least in web tech) seems to do print statement debugging. I have tried and failed to get others interested in using my workflow.

I generally agree that it’s the best way to start understanding a system. Breaking on an interesting line of code during a test run and studying the call stack that got me there is infinitely easier than trying to run the code forwards in my head.

Young grugs: learning this skill is a minor superpower. Take the time to get it working on your codebase, if you can.

replies(48): >>44305342 #>>44305375 #>>44305388 #>>44305397 #>>44305400 #>>44305414 #>>44305437 #>>44305534 #>>44305552 #>>44305628 #>>44305806 #>>44306019 #>>44306034 #>>44306065 #>>44306133 #>>44306145 #>>44306181 #>>44306196 #>>44306403 #>>44306413 #>>44306490 #>>44306654 #>>44306671 #>>44306799 #>>44307053 #>>44307204 #>>44307278 #>>44307864 #>>44307933 #>>44308158 #>>44308299 #>>44308373 #>>44308540 #>>44308675 #>>44309088 #>>44309822 #>>44309825 #>>44309836 #>>44310156 #>>44310430 #>>44310742 #>>44311403 #>>44311432 #>>44311683 #>>44312050 #>>44312132 #>>44313580 #>>44315651 #
demosthanos ◴[18 Jun 25 00:16 UTC] No.44305400[source]▶
There was a good discussion on this topic years ago [0]. The top comment shares this quote from Brian Kernighan and Rob Pike, neither of whom I'd call a young grug:

> As personal choice, we tend not to use debuggers beyond getting a stack trace or the value of a variable or two. One reason is that it is easy to get lost in details of complicated data structures and control flow; we find stepping through a program less productive than thinking harder and adding output statements and self-checking code at critical places. Clicking over statements takes longer than scanning the output of judiciously-placed displays. It takes less time to decide where to put print statements than to single-step to the critical section of code, even assuming we know where that is. More important, debugging statements stay with the program; debugging sessions are transient.

I tend to agree with them on this. For almost all of the work that I do, this hypothesis-logs-exec loop gets me to the answer substantially faster. I'm not "trying to run the code forwards in my head". I already have a working model for the way that the code runs, I know what output I expect to see if the program is behaving according to that model, and I can usually quickly intuit what is actually happening based on the incorrect output from the prints.

[0] The unreasonable effectiveness of print debugging (349 points, 354 comments) April 2021 https://news.ycombinator.com/item?id=26925570

replies(25): >>44305453 #>>44305548 #>>44305864 #>>44305954 #>>44305964 #>>44306045 #>>44306147 #>>44306151 #>>44306280 #>>44306332 #>>44306505 #>>44307171 #>>44307364 #>>44307835 #>>44307858 #>>44307897 #>>44307934 #>>44308016 #>>44308282 #>>44308302 #>>44309738 #>>44311312 #>>44312123 #>>44314764 #>>44322638 #
johnfn ◴[18 Jun 25 02:56 UTC] No.44306280[source]▶
On the other hand, John Carmack loves debuggers - he talks about the importance of knowing your debugging tools and using them to step through a complex system in his interview with Lex Friedman. I think it's fair to say that there's some nuance to the conversation.

My guess is that:

- Debuggers are most useful when you have a very poor understanding of the problem domain. Maybe you just joined a new company or are exploring an area of the code for the first time. In that case you can pick up a lot of information quickly with a debugger.

- Print debugging is most useful when you understand the code quite well, and are pretty sure you've got an idea of where the problem lies. In that case, a few judicious print statements can quickly illuminate things and get you back to what you were doing.

replies(12): >>44306444 #>>44306679 #>>44306784 #>>44306846 #>>44306928 #>>44307358 #>>44307480 #>>44309591 #>>44310071 #>>44310380 #>>44313608 #>>44324296 #
1. tcoff91 ◴[18 Jun 25 05:37 UTC] No.44306928[source]▶
Adding print statements sucks when you are working on native apps and you have to wait for the compiler and linker every time you add one. Debuggers hands down if you are working on something like C++ or Rust. You can add tracepoints in your debugger if you want to do print debugging in native code.

In scripting languages print debugging makes sense especially when debugging a distributed system.

Also logging works better than breaking when debugging multithreading issues imo.

I use both methods.

replies(2): >>44307198 #>>44309960 #
2. mort96 ◴[18 Jun 25 06:36 UTC] No.44307198[source]▶
How long are you realistically "waiting for the compiler and linker"? 3 seconds? You're not recompiling the whole project after all, just one source file typically

If I wanna use a debugger though, now that means a full recompile to build the project without optimizations, which probably takes many minutes. And then I'll have to hope that I can reproduce the issue without optimizations.

replies(5): >>44307426 #>>44307475 #>>44307612 #>>44309097 #>>44309157 #
3. MaulingMonkey ◴[18 Jun 25 07:21 UTC] No.44307426[source]▶
> How long are you realistically "waiting for the compiler and linker"? 3 seconds?

I've worked on projects where incremental linking after touching the wrong .cpp file will take several minutes... and this is after I've optimized link times from e.g. switching from BFD to Gold, whereas before it might take 30 minutes.

A full build (from e.g. touching the wrong header) is measured in hours.

And that's for a single configuration x platform combination, and I'm often the sort to work on the multi-platform abstractions, meaning I have it even worse than that.

> If I wanna use a debugger though, now that means a full recompile to build the project without optimizations

You can use debuggers on optimized builds, and in fact I use debuggers on optimized builds more frequently than I do unoptimized builds. Granted, to make sense of some of the undefined behavior heisenbugs you'll have to understand disassembly, and dig into the lower level details when the high level ones are unavailable or confused by optimization, but those are learnable skills. It's also possible to turn off optimizations on a per-module basis with pragmas, although then you're back into incremental build territory.

replies(1): >>44311512 #
4. draven ◴[18 Jun 25 07:30 UTC] No.44307475[source]▶
I work on Scala projects. Adding a log means stopping the service, recompiling and restarting. For projects using Play (the biggest one is one of them) that means waiting for the hot-reload to complete. In both cases it easily takes at least 30s on the smallest projects, with a fast machine. With my previous machine Play's hot reload on our biggest app could take 1 to 3mn.

I use the debugger in Intellij, using breakpoints which only log some context and do not stop the current thread. I can add / remove them without recompiling. There's no interruption in my flow (thus no excuse to check HN because "compiling...")

When I show this to colleagues they think it's really cool. Then they go back to using print statements anyway /shrug

replies(1): >>44311537 #
5. writebetterc ◴[18 Jun 25 07:56 UTC] No.44307612[source]▶
>How long are you realistically "waiting for the compiler and linker"? 3 seconds? You're not recompiling the whole project after all, just one source file typically

10 minutes.

>If I wanna use a debugger though, now that means a full recompile to build the project without optimizations, which probably takes many minutes.

Typically always compile with debug support. You can debug an optimized build as well. Full recompile takes up to 45 minutes.

The largest reason to use a debugger is the time to recompile. Kinda, I actually like rr a lot and would prefer that to print debugging.

replies(1): >>44307839 #
6. mort96 ◴[18 Jun 25 08:27 UTC] No.44307839{3}[source]▶
10 minutes for linking? The only projects I've touched which have had those kinds of link times have been behemoths like Chromium. That must absolutely suck to work with.

Have you tried out the Mold linker? It might speed it up significantly.

> You can debug an optimized build as well.

Eh, not really. Working with a binary where all variables are optimized out and all operators are inlined is hell.

replies(1): >>44308095 #
7. writebetterc ◴[18 Jun 25 09:12 UTC] No.44308095{4}[source]▶
>10 minutes for linking? The only projects I've touched which have had those kinds of link times have been behemoths like Chromium. That must absolutely suck to work with.

I don't know the exact amounts of time per phase, but you might change a header file and that will of course hurt you a lot more than 1 translation unit.

> Eh, not really. Working with a binary where all variables are optimized out and all operators are inlined is hell.

Yeah, but sometimes that's life. Reading the assembly and what not to figure things out.

>That must absolutely suck to work with.

Well, you know, I also get to work on something approximately as exciting as Chromium.

replies(1): >>44308130 #
8. mort96 ◴[18 Jun 25 09:19 UTC] No.44308130{5}[source]▶
> I don't know the exact amounts of time per phase, but you might change a header file and that will of course hurt you a lot more than 1 translation unit.

Yeah, which is why I was talking about source files. I was surprised that changing 1 source file (meaning re-compiling that one source file and then re-linking the project) takes 10 minutes. If you're changing header files then yeah it's gonna take longer.

replies(1): >>44308200 #
9. writebetterc ◴[18 Jun 25 09:30 UTC] No.44308200{6}[source]▶
FWIW, I've heard from people who know this stuff that linking is actually super slow for us :). I also wanted to try out mold, but I couldn't manage to get it to work.
10. catlifeonmars ◴[18 Jun 25 11:58 UTC] No.44309097[source]▶
> How long are you realistically “waiting for the compiler and linker”

Debugging kernel module issues on AL2 on bare metal ec2 with kprint. Issue does not reproduce in qemu. It happens

11. pjc50 ◴[18 Jun 25 12:08 UTC] No.44309157[source]▶
> How long are you realistically "waiting for the compiler and linker"? 3 seconds?

This is the "it's one banana Michael, how much could it cost, ten dollars?" of tech. I don't think I've ever worked on a nontrivial C++ project that compiled in three seconds. I've worked in plenty of embedded environments where simply the download-and-reboot cycle took over a minute. Those are the places where an interactive debugger is most useful .. and also sometimes most difficult.

(at my 2000s era job a full build took over an hour, so we had a machine room of ccache+distcc to bring it down to a single digit number of minutes. Then if you needed to do a full place and route and timing analysis run that took anything up to twelve hours. We're deep into "uphill both ways" territory now though)

replies(1): >>44310078 #
12. eptcyka ◴[18 Jun 25 13:59 UTC] No.44309960[source]▶
Intoruding logging can actually hide concurrency bugs.
replies(1): >>44311734 #
13. mort96 ◴[18 Jun 25 14:12 UTC] No.44310078{3}[source]▶
> I don't think I've ever worked on a nontrivial C++ project that compiled in three seconds.

No C++ project compiles in 3 seconds, but your "change a single source file and compile+link" time is often on the order of a couple of seconds. As an example, I'm working on a project right now where a clean build takes roughly 30 seconds (thanks to recent efforts to improve header include hygiene and move stuff from headers into source files using PIMPL; it was over twice that before). However, when I changed a single source file and ran 'time ninja -C build' just now, the time to compile that one file and re-link the project took just 1.5 seconds.

I know that there are some projects which are much slower to link, I've had to deal with Chromium now and then and that takes minutes just to link. But most projects I've worked with aren't that bad.

14. norir ◴[18 Jun 25 16:57 UTC] No.44311512{3}[source]▶
I understand you are likely doing this for shiny rocks, but life is too short to spend on abominable code bases like this.
replies(1): >>44313290 #
15. norir ◴[18 Jun 25 17:00 UTC] No.44311537{3}[source]▶
This reminds me why I abandoned scala. That being said, even a small sbt project can cold boot in under 10 seconds on a six year old laptop. I shudder to think of the bloat in play if 30 seconds is the norm for a hot reload.
16. tcoff91 ◴[18 Jun 25 17:26 UTC] No.44311734[source]▶
Yeah this is true, it can change the timing. But setting breakpoints or even just running in a debugger or even running a debug build at all without optimizations can also hide concurrency bugs. Literally anything can hide concurrency bugs.

Concurrency bugs just suck to debug sometimes.

17. PaulDavisThe1st ◴[18 Jun 25 21:07 UTC] No.44313290{4}[source]▶
Even in my open source income-generating code base (ardour.org) a complete build takes at least 3 minutes on current Apple ARM hardware, and up to 9mins on my 16 core Ryzen. It's quite a nice code base, according to most people who see it.

Sometimes, you just really do need hundreds of thousands of lines of C++ (or equivalent) ...