←back to thread

Delete tests

(andre.arko.net)
125 points mooreds | 1 comments | | HN request time: 0s | source
Show context
recursivedoubts ◴[] No.45071410[source]
One of the most important things you can do is move your tests up the abstraction layers and away from unit tests. For lack of a better term, to move to integration tests. End-to-end tests are often too far from the system to easily understand what's wrong when they break, and can overwhelm a development org. Integration tests (or whatever you want to call them) are often the sweet spot: not tied to a particular implementation, able to survive fairly significant system changes, but also easy enough to debug when they break.

https://grugbrain.dev/#grug-on-testing

replies(11): >>45071535 #>>45071726 #>>45071751 #>>45071944 #>>45072117 #>>45072123 #>>45072158 #>>45072321 #>>45072494 #>>45074365 #>>45080184 #
RHSeeger ◴[] No.45071726[source]
Integration tests and Unit tests are different tools; and each has their place and purpose. Using one "instead" of the other is a mistake.
replies(8): >>45072079 #>>45072176 #>>45072722 #>>45072873 #>>45073135 #>>45074394 #>>45080460 #>>45093392 #
simianwords ◴[] No.45072176[source]
Wow I hate this dogmatism. It is indeed better to use one instead of the other. Let’s stop pretending all are equally good and we need every type of test.

Sometimes you just don’t need unit tests and it’s okay to admit it and work accordingly.

replies(3): >>45072205 #>>45072404 #>>45072431 #
imiric ◴[] No.45072404[source]
You claim it's dogmatism, yet do the same thing in reverse. (:

Unit and integration tests test different layers of the system, and one isn't inherently better or more useful than the other. They complement each other to cover behavior that is impossible to test otherwise. You can't test low-level functionality in integration tests, just as you can't test high-level functionality in unit tests.

There's nothing dogmatic about that statement. If you disagree with it, that's your prerogative, but it's also my opinion that it is a mistake. It is a harmful mentality that makes code bases risky to change, and regressions more likely. So feel free to adopt it in your personal projects if you wish, but don't be surprised if you get push back on it when working in a team. Unless your teammates think the same, in which case, good luck to you all.

replies(1): >>45072649 #
tsimionescu ◴[] No.45072649[source]
The problem with this line of argument is that, in general, high level behavior (covered by integratuon tests) is dependent on low level behavior. So if your code is ascertained to work at the high level, you also know that it must be working at the lower level too. So, integration tests also tell you if your component works at a low level, not just a high level.

The converse is not true, however. It's perfectly possible for individual components to "work" well, but to not do the right thing from a high level perspective. Say, one component provides a good fast quicksort function, but the other component requires a stable sort to work properly - each is OK in isolation, but you need an integration test to figure out the mistake.

Unit tests are typically good scaffolding. They allow you to test bits of your infrastructure as you're building it but before it's ready for integration into the larger project. But they give you realtively little assurance at the project level, and are not worth it unless you're pretty sure you're building the right thing in the first place.

replies(3): >>45073029 #>>45073085 #>>45073466 #
imiric ◴[] No.45073029[source]
> So if your code is ascertained to work at the high level, you also know that it must be working at the lower level too.

No, that is not guaranteed.

Integration and E2E tests can only cover certain code paths, because they depend on the input and output from other systems (frontend, databases, etc.). This I/O might be crafted in ways that never trigger a failure scenario or expose a bug within the lower-level components. This doesn't mean that the issue doesn't exist—it just means that you're not seeing it.

Furthermore, the fact that, by their nature, integration and E2E tests are often more expensive to setup and run, there will be fewer of them, which means they will not have full coverage of the underlying components. Another issue is that often these tests, particularly E2E and acceptance tests, are written only with a happy path in mind, and ignore the myriad of input that might trigger a failure in the real world.

Another problem with your argument is that it ignores that tests have different audiences. E2E and acceptance tests are written for the end user; integration tests are written for system integrators and operators; and unit tests are written for users of the API, which includes the author and other programmers. If you disregard one set of tests, you are disregarding that audience.

To a programmer and maintainer of the software, E2E and acceptance tests have little value. They might not use the software at all. What they do care about is that the function, method, object, module, or package, does what says on the tin; that it returns the correct output when given a specific input; that it's performant, efficient, well documented, and so on. These users matter because they are the ones who will maintain the software in the long run.

So thinking that unit tests are useless because they're a chore to maintain is a very shortsighted mentality. Instead, it's more beneficial to see them as guardrails that make your future work easier, by giving you the confidence that you're not inadvertently breaking an API contract whenever you make a change, even when all higher-level tests remain green across the board.

replies(2): >>45073178 #>>45080787 #
simianwords ◴[] No.45080787{4}[source]
> So thinking that unit tests are useless because they're a chore to maintain is a very shortsighted mentality. Instead, it's more beneficial to see them as guardrails that make your future work easier, by giving you the confidence that you're not inadvertently breaking an API contract whenever you make a change, even when all higher-level tests remain green across the board.

This is the kind of dogmatism I want people to understand. I’m not saying unit tests are useless but they have very narrow use, in units that encompass slightly complicated logic. Most of us write classes that just have a few for loops, if conditions, metrics and a few transformations. The overhead of writing a unit tests where, mocking all external services and continuously maintaining them when every small code change causes unit tests to break (false positives) is pretty high.

replies(1): >>45081993 #
1. imiric ◴[] No.45081993{5}[source]
> Most of us write classes that just have a few for loops, if conditions, metrics and a few transformations.

You're describing code. At what point does code become "worthy" of a unit test? How do you communicate this to your team members? This type of ambiguity introduces friction and endless discussions in code reviews, to the point that abiding to the convention that all code should be unit tested whenever possible is a saner long-term strategy. This doesn't have to be a strict rule, but it makes sense as a general convention. Besides, these days with LLMs, writing and maintaining unit tests doesn't have to be a chore anymore. It's one thing the tech is actually reasonably good at.

What I think we fundamentally disagree about is the value of unit tests. That small function with a few for loops and if conditions still has users, which at the end of the day might be only yourself. You can't be sure that it's working as intended without calling it. You can do this either manually; automatically by the adjacent code that calls it, whether that's within an integration/E2E test or in production; or with automated unit tests. Out of those options, automated unit tests are the ones that provide the highest degree of confidence, since you have direct control over its inputs and visibility of its outputs. Everything else has varying degrees of uncertainty, which carries a chance of exposing an issue to end users.

Now, you might be fine with that uncertainty, especially if you're working on a solo project. But this doesn't mean that there's no value in having extensive coverage from unit tests. It just means that you're willing to accept a certain level of uncertainty, willing to tradeoff confidence for convenience of not having to write and maintain code that you personally don't find valuable, and willing to accept the risk of exposing issues to end users.