Parse, Don't Validate (2019)

I’d argue that the more experience you get the more you write code for other people which involves adding lots of tooling, tests, etc. Even if the code works the first time, a more senior dev will make sure others have a “pit of success” they can fall into. This involves a lot more than just some “unit tests as an afterthought to keep the coverage up.”

Keeping the code simple, finding the right abstractions, untangling coupling, gets the most bang for the buck. See the “beyond pep8” talk for a enlightened perspective.

That said, lightweight testing and tools like pyflakes to prevent egregious errors helps an experienced dev write very productively. Typing helps the most with large, venerable projects with numerous devs of differing experience levels.

It immediately tells me that they've never worked on large software projects, and if they have they haven't worked on ones that lasted more than a few months.

I apologize to folks reading this for my rather aggressive tone but I've been writing software for a long time in numerous languages, and people with the unit tests as an afterthought attitude are typically rather arrogant in fool hardy.

The most recent incarnation I've encountered is the hotshot data scientist who did okay in a few Kaggle competitions using Jupyter notebooks, and thinks they can just write software the way they did for the competitions with no test of any kind.

I had one of these on my team recently and naturally I had to do 95% of the work to turn anything he produced into a remotely decent product. I couldn't even get the guy to use nbdev, which would have allowed him to use Jupyter to write tested, documented, maintainable code.

I disagree. I've started using types from the ground up and it helps almost equally at every stage of the game. Also I aggressively rely on autocomplete for methods. It's faster this way than usual "dynamic" or "pythonic" python.

Part of it might be exactly because writing my datatypes first helps me think about the right abstractions.

The big win with python is maybe 2-10% of functions, I just want to punt and use a dict. But I have shifted >80% of what used to be dicts to Models/dataclasses and it's so much faster to write and easier to debug.

On certain backend code where I am able to do unit tests, they do catch the occasional edge case logic error but not at a rate that makes me concerned about only checking them in some time after the original code, which I'll have already tested myself in real use as I went along.

Also, what makes you think I’m not aware of datatypes? Currently working eight hours a day on Django models.

In short, there are choices besides, “I alone have to do all the hard work.”

I was more experienced with predictive algorithms and deep learning than any of the data scientists at the company but because they were brought in from an acquisition of a company that had an undeserved reputation due to a loose affiliation with MIT, they were treated like magicians while the rest of us were treated like blacksmiths.

I had the choice and I made the choice to leave. And of course I raised hell with the bosses about them not writing remotely production quality code that required extensive refactoring.

And yes I was paid to do the work but the work occupied time that I could have spent working on the other projects I had that were more commercially successful but less sexy to Silicon Valley VCs who look at valuations based on other companies' newest hottest product.

I don't deny it. Join the cult of Static Python. We have cookies! And lower stress levels!

I usually wrap that spiel with my caveat "this depends greatly on your neurotype, style, environment, and other things." I have ADHD and my brain struggles with keeping bits of state in memory, so having to remember the type of every variable without my IDE tracking it for me is a huge performance drain.

However, I would contend even if your neurotype supported that mental workflow... it isn't actually better. Humans on average can handle 7 +/- 2 "pieces" of information in focus. Why spend any of your precious half-dozen pieces of salient consciousness on something a machine is really good at doing?

Opinion: This is actually a symptom of what is (imho) a pervasive problem lodged deep in the collective consciousness of software dev: OOP with fine-grained objects. I blame (early) Java in large part for exacerbating this mentality. Encapsulation of state with mutator methods in particular. It sprays state all over the application, encourages mutation in place over immutability, coupling, validating-not-parsing, and makes it nigh-well impossible to write good tests.

It's really hard to write objects that enforce all invariants under every mutation. And when you have state strewn everywhere, it's impossible to test every nook and cranny. The combinatorial space explodes.

Objects are helpful for encapsulating state when they are course-grained, mutations are atomic, coupling occurs in one place, state changes are auditable, and the entire state can be replayed/set at once, to enable mock tests and subsystem integration tests. AKA, things like databases, reactors, and persistent data structures.

Also since the tools are immature and bolted on afterward in Python, I think it's even a bit worse than it would be in something decent like C#.