←back to thread

Delete tests

(andre.arko.net)
125 points mooreds | 1 comments | | HN request time: 0.226s | source
1. imiric ◴[] No.45072028[source]
I'm a big believer in the utility of tests, and I do think the author has a point. There is a time and place when a test is not useful, and should be deleted.

However...

> If the future bug occurs, fix it and write a new test that doesn’t flake. Today, delete the tests.

How is this different from simply fixing the flaky test today?

Tests are code, and can also incur technical debt. The reason the test is flaky is likely because nobody is willing to take the time to address it properly. Sometimes it requires a refactoring of the SUT to allow making the test more reliable. Sometimes the test itself is too convoluted and difficult to change. All of this is chore work, and is often underappreciated. Nobody got promoted or celebrated for fixing something that is an issue a random percentage of times. After all, how do we know for sure that it's permanently fixed? Only time will tell.

But the flaky test might still deliver confidence and be valuable when it does run successfully. So deleting it would bring more uncertainty. That doesn't seem like a fair tradeoff for removing an annoyance. The better approach would be to deal with the annoyance.

> What if your tests are written so that a one line code change means updating 150 tests?

That might be a sign that the tests are too brittle, and too "whiteboxy". So fix the tests.

That said, there are situations when a change does require updating many tests. These are usually large refactors or major business logic changes. This doesn't mean that the tests are and won't be useful. It's just a side effect of the change. Tests are code, so fix the tests.

I've often heard negativity around unit tests, from programmers who strongly believe that more utility comes from integration tests (the inverted test pyramid, etc.). One of the primary reasons is this belief that unit tests slow you down because they need to be constantly updated. This is a harmful mentality, coming from a place of laziness.

Tests are code, and require maintenance just as well. Unit tests in particular are tightly coupled to the SUT, which makes them require maintenance more frequently. There should also be more unit tests than other types, adding more maintenance burden. But none of these are reasons to not write unit tests, and codebases without them are more difficult to change, and more susceptible to regressions.

> What if your tests take so long to run that you can’t run them all between merges, and you start skipping some?

That is an organizational problem. Label your tests by category (unit, integration, E2E), and provide quick ways to run a subset of them. During development, you can run the quick tests for a sanity check, while the more expensive tests run in CI.

There's also the problem of long test suites because the tests are inefficient.

Again: *fix the tests*.

> Even worse, what if your business requirements have changed, and now you have thousands of lines of tests failing because they test the wrong thing?

That is a general maintenance task. Would you say the same because you had to update a library that depended on the previous business logic? Would you simply delete the library because it would take a lot of effort to update it?

No?

Then *fix the tests*. :)