←back to thread

Unit tests as documentation

(www.thecoder.cafe)
94 points thunderbong | 1 comments | | HN request time: 0.216s | source
1. lihaoyi ◴[] No.41875694[source]
I make heavy use of this idea in many of my open source projects. I've tried a variety of approaches:

* ScalaSql, where the reference docs (e.g. https://github.com/com-lihaoyi/scalasql/blob/main/docs/refer...) are generated by running unit tests (e.g. https://github.com/com-lihaoyi/scalasql/blob/53cbad77f7253f3...)

* uPickle, where the documentation site (https://com-lihaoyi.github.io/upickle/#GettingStarted) is generated by the document-generator which has syntax to scrape (https://github.com/com-lihaoyi/upickle/blob/004ed7e17271635d...) the unit tests without running them (https://github.com/com-lihaoyi/upickle/blob/main/upickle/tes...)

* OS-Lib, where the documentation examples (e.g. https://github.com/com-lihaoyi/os-lib?tab=readme-ov-file#osr...) are largely manually copy-pasted from the unit tests (e.g. https://github.com/com-lihaoyi/os-lib/blob/9e7efc36355103d71...) into the readme.md/adoc

It's a good idea overall to share unit tests and documentation, but there is a lot of subtlety around how it must be done. Unit tests and documentation have many conflicting requirements, e.g.

* Unit tests prefer thoroughness to catch unintuitive edge cases whereas documentation prefers highlighting of key examples and allowing the reader to intuitively interpolate

* Unit tests examples prefer DRY conciseness whereas documentation examples prefer self-contained-ness

* Unit tests are targeted at codebase internal developers (i.e. experts) whereas documentation is often targeted at external users (i.e. non-experts)

These conflicting requirements mean that "just read the unit tests" is a poor substitute for documentation. But there is a lot of overlap, so it is still worth sharing snippets between unit tests and examples. It just needs to be done carefully and with thought given handling the two sets of conflicting requirements