Parse, Don't Validate (2019)

For sure. I do most of my work in situations of high volatility of domain and requirements and relatively high risk. (E.g., startups, projects in new areas.)

Static typing really appeals to me on a personal level. I enjoy the process of analysis it requires. I love the notion of eliminating whole classes of bugs. It feels way more tidy. I took Odersky's Scala class for fun and loved it.

But in practice, they're just a bad match for projects where the defining characteristic is unstable ground. They force artificial clarity when the reality is murky. And they impose costs that pay off in the long run, which only matters if the project has a long run. If I'm building something where we don't know where we're going, I'll reach for something like Python or Ruby to start.

This has been brought home to me by doing interviews recently. I have a sample problem that we pair on for an hour or so; there are 4 user stories. It involves a back end and a web front end. People can use any tools they want. My goal isn't to get particular things done; it's to see them at their best.

After doing a couple dozen, I'm seeing a pattern: developers using static tooling (e.g., Java, TypeScript) get circa half as much done as people using dynamic tooling (Python, plain JS). In the time when people in static contexts are still defining interfaces and types, people using dynamic tools are putting useful things on the page. Making a change in the static code often requires multiple tweaks in situations where it's one change in the dynamic code. It makes the extra costs of static tooling really obvious.

That doesn't harm the static-language interviewees, I should underline. The goal is to see how they work. But it was interesting to see that it wasn't just me feeling the extra costs. And those costs are only worth paying when they create payoffs down the road.

Great comment. There are no silver bullets. I am Team static typing, but recognize how heavy of a burden would be to start a purely exploratory development in Rust or Java. It just "cuts your wings" in the name of correctness... well some times it is useful to have the ability to start with a technically incorrect implementation that anyways only fails in a corner case that is not your main point of research.

On the other hand, as the initial code grows and grows, the cost of moving it all to a saner language grows too... discouraging a rewrite. So we end up with very complex production software that started as dynamic and is still dynamic.

I have had the exact same experience. There's lots of utility in statically typed languages. They're great if your problem space is well defined. With respect to type checking, it's like a jig in wood or metal working. You trade flexibility for correctness.

When the problem space is less well defined the type-related boiler plate adds a lot of friction. It's not impossible to overcome that friction but it slows down progress. When you're under a tight deadline development velocity is often more valuable than absolute correctness or even overall runtime efficiency.

An delivered product that works is usually more valuable than an undelivered product that's more "correct" or efficient. A development project is just a cost (for various values of cost) until it ships.

Perhaps the Goldilocks mixture will be languages that allow type annotations but don't require it (e.g. Typescript, Elixir, Racket, and I think this is how Python's works).

This is the best comment on the subject and should be at the top rather the current dogmatic ones.

"type-related boiler plate"

That phrase makes me sad. Mainstream languages have a lot of scope for improvement in their type systems.

Yeah, I've been using Python's gradual typing for a while. It's not perfect, but I'm excited for the possibilities. But the real test is to see what it's like on a large, long-lived project, so I'm keeping any open mind. I figure if that doesn't work fully, it'll still be a nice step toward things that can be pulled out as isolated services.

Definitely. And for me the early stages of a product are often about buying information. "Users say they want X, so let's ship X and see." Key to exploring a product space is tight feedback loops between having an idea and seeing what people really do. It's only once I have enough active users (especially active paying users) to justify the project that I have some confidence about what "long term" really means for the code base.

> But in practice, they're just a bad match for projects where the defining characteristic is unstable ground. They force artificial clarity when the reality is murky.

The rebuttal to this is provided by another post from Alexis King: https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-typ...

> This story sounds compelling, but it isn’t true. The flaw is in the premise: static types are not about “classifying the world” or pinning down the structure of every value in a system. The reality is that static type systems allow specifying exactly how much a component needs to know about the structure of its inputs, and conversely, how much it doesn’t.

Static typing helps you to safely model the parts of the system that you actually do know about, while allowing you to leave the unknown parts loosely defined. And when business requirements change, the compiler helps you to make those changes in a safe, guided way. This is why people often report the experience of doing large refactorings driven by compiler error messages, then running and finding that the change works correctly on the first try. A confidence few report in dynamically-typed languages.

> After doing a couple dozen, I'm seeing a pattern: developers using static tooling (e.g., Java, TypeScript) get circa half as much done as people using dynamic tooling (Python, plain JS).

I hope you understand and accept the fact that interview code challenges are not very representative of real-world software engineering. On-the-spot interview code is throwaway; real production code is not.

In this case, the interview problem was a simplified version of a thing the team gets asked to do. So I believe it was as representative as possible for an hour's coding.

I disagree that "real production code" is not throwaway. In situations of high requirements volatility, a great deal of production code ends up getting thrown away, because either a) it's intended to be a short-lived way to learn about the domain, or b) it's built on an assumption that turns out to be incorrect. And until the project hits economic sustainability, a lot of assumptions necessarily get made. It's only once the project is economically sustainable that we have some confidence that the code will be around for the long haul.

If your point is that most people are using statically typed languages wrong for high-volatility situations, I'll take your word for it. There were definitely times Typescript users seemed to be unnecessarily constraining things, for example. But I'm not sold that the more elaborate approach of being thoughtfully and explicitly untyped for a project's early stages for the sake of prototyping in a statically typed language would yield much practical gain.

> In this case, the interview problem was a simplified version of a thing the team gets asked to do. So I believe it was as representative as possible for an hour's coding.

That's exactly my point–code written under pressure during an hour-long code challenge will never be representative of code written during normal day-to-day coding. Even if the problem domain is similar to a real-world problem.

> I disagree that "real production code" is not throwaway. In situations of high requirements volatility, a great deal of production code ends up getting thrown away

But it's not written to be thrown away an hour later. That's not really the kind of 'production code' that any reasonable person would be talking about in this thread. If you're talking about exploratory data analysis, data science or whatever, sure, you can consider it 'production code', but it's not the product of the craft of software engineering, it's more about running quick experiments.

> I'm not sold that the more elaborate approach of being thoughtfully and explicitly untyped for a project's early stages for the sake of prototyping in a statically typed language would yield much practical gain.

There are many different kinds of projects. And people who work with good statically typed languages can prototype quite well using types to drive their domain modelling. It's a different way of working. You should read some of what Scott Wlaschin has written to get an idea.

> but it's not the product of the craft of software engineering

Sure? This seems pretty no-true-Scotsman to me. If you want to define some the things I need to do as not real software, feel free. But they're still things my team has to get done. To me there's a spectrum of domain stability on which the code rests. It's frequently not possible to know in advance which bits will be long lived and which won't be. So whether or not one of htose bits is the true "craft of software engineering" in somebody's eyes has very little interest to me. Especially when it's an anonymous rando.

> people who work with good statically typed languages can prototype quite well using types to drive their domain modelling.

I can certainly believe that's possible. Hopefully some of them will interview with me so I can see how they work. I'm just reporting the data I have.

> that's not really the kind of 'production code' that any reasonable person

In startup contexts I will quite often try an experiment that is quick to build. A few hours is not at all uncommon. That you consider me unreasonable is a sign I can better spend my time elsewhere.