←back to thread

Unit tests as documentation

(www.thecoder.cafe)
94 points thunderbong | 2 comments | | HN request time: 0.82s | source
Show context
danjl ◴[] No.41872129[source]
Why just unit tests? Integration tests seem much more valuable as documentation of what the users will do in the app. Unit tests have limited benefits overall, and add a bunch of support time, slowing down development. If you have good (90%+) coverage just from integration tests, you are likely doing 90%+ coverage of the unit tests at the same time, without the extra effort or support burden. You can use the same reasoning to describe the benefits for understanding the code, you get a clear understanding of the important usage cases, plus you get the unit-level "documentation" for free.
replies(4): >>41872167 #>>41872168 #>>41872352 #>>41872618 #
1. evil-olive ◴[] No.41872352[source]
unit vs integration tests is not an either/or. you need both, and in appropriate coverage amounts.

a common way to think about this is called the "test pyramid" - unit tests at the base, supporting integration tests that are farther up the pyramid. [0]

roughly speaking, the X-axis of the pyramid is number of test cases, the Y-axis is number of dependencies / things that can cause a test to fail.

as you travel up the Y-axis, you get more "lifelike" in your testing...but you also generally increase the time & complexity it takes to find the root-cause of a test failure.

many times I've had to troubleshoot a failure in an integration test that is trying to test subsystem A, and it turns out the failure was caused by unrelated flakiness in subsystem B. it's good to find that flakiness...but it's also important to be able to push that testing "down the pyramid" and add a unit test of subsystem B to prevent the flakiness from reoccurring, and to point directly at the problem if it does.

> Unit tests have limited benefits overall, and add a bunch of support time, slowing down development

unit tests, _when done poorly_, have limited benefits, require additional maintenance, and slow down development.

integration tests can also have limited benefits, require additional maintenance, and slow down development time, _when done poorly_.

testing in general, _when done well_, increases development velocity and improves product quality in a way that completely justifies the maintenance burden of the additional code.

0: https://martinfowler.com/articles/practical-test-pyramid.htm...

replies(1): >>41876521 #
2. BeetleB ◴[] No.41876521[source]
> unit vs integration tests is not an either/or. you need both, and in appropriate coverage amounts.

Agreed. But I also agree with the commenter that for documentation purposes, integration tests are an order of magnitude more useful.

> a common way to think about this is called the "test pyramid" - unit tests at the base, supporting integration tests that are farther up the pyramid.

I used to be a believer in that pyramid, but my experience has shown me it depends on the project. Wherever it's feasible (i.e. doesn't involve long test times), I've found integration tests to be far more useful than unit tests. I've had experiences where I'd do a project and have really high unit test coverage, only to unveil fairly trivial bugs. The reverse hasn't happened - if I start a project with solid integration tests, I almost never encounter trivial bugs.

Generally, I now write integration tests and mock away time consuming/resource heavy parts (e.g. network calls, DB calls, etc). Better for documentation. Better for testing.