In the example the author walks through, a cleaner way would be to have the second function take the Options as a parameter and decouple those two functions. You can then test both in isolation.
Note that I said test doubles. Mocks are a bit over specific - they are about verifying functions are called at the right time with the right arguments, but the easy ability to set return values makes it easy to abuse them for other things (this abuse is good, but it is still abuse of the intent).
In this case you want a fake: a smart service that when you are in a test setups a temporary directory tree that contains all the files you need in the state that particular test needs, and destroys that when the test is done (with an optional mode to keep it - useful if a test fails to see debug). Depending on your situation you may need something for network services, time, or other such things. Note that in most cases a filesystem itself is more than fast enough to use in tests, but you need isolation from other tests. There are a number of ways to create this fake, it could override open, or it could just be a GetMyProgramDir function that you override are two that I can think of.
Valgrind is a mock of standard library/OS functions and I think its existence is a good thing. Simulating OOM is also only possible by mocking stuff like open.
I like Hynek Schlawak's 'Don’t Mock What You Don’t Own' [1] phrasing, and while I'm not a fan of adding too many layers of abstraction to an application that hasn't proved that it needs them, the one structure I find consistently useful is to add a very thin layer over parts that do I/O, converting to/from types that you own to whatever is needed for the actual thing.
These layers should be boring and narrow (for example, never mock past validation you depend upon), doing as little conversion as possible. You can also rephrase the general purpose open()-type usage into application/purpose-specific usages of that.
Then you can either unittest.mock.patch these or provide alternate stub implementations for tests in a different way, with this this approach also translating easily to other languages that don't have the (double-edged sword) flexibility of Python's own unittest.mock.
It does. And this is exactly the problem, here!
> TFA: The thing we want to avoid is opening a real file
No! No, no, no! You do not 'want to avoid opening a real file' in a test.
It's completely fine to open a real file in a test! If your code depends on reading input files, then your test should include real input files in it! There's no reason to mock any of this. All of this stuff is easy to set up in any unit test library worth it's salt.
That is mostly wrong.
Valgrind wraps syscalls. For the most part it just checks the arguments and records any reads or writes to memory. For a small number of syscalls it replaces the syscall rather than wrapping it (for instance calls like getcontext where it needs to get the context from the VEX synthetic CPU rather than the real CPU).
Depending on the tool it can also wrap or replace libc and libpthread functions. memcheck will replace all allocation functions. DRD and Helgrind wrap all pthread functions.
That's okay for testing some branches of your code. But not all. I don't want to have to actually crash my hard drive to test that I am properly handling hard drive crashes. Mocking[1] is the easiest way to do that.
[1] For some definition of mock. There is absolutely no agreement found in this space as to what the terms used mean.
$ cat test.c
void main (void) {
malloc (1000);
}
$ make test
cc test.c -o test
$ valgrind --leak-check=full --show-leak-kinds=all -s ./test
Memcheck, a memory error detector
Command: ./test
HEAP SUMMARY:
in use at exit: 1,000 bytes in 1 blocks
total heap usage: 1 allocs, 0 frees, 1,000 bytes allocated
1,000 bytes in 1 blocks are still reachable in loss record 1 of 1
at 0x483877F: malloc (vg_replace_malloc.c:307)
by 0x109142: main (in test.c:2)
LEAK SUMMARY:
definitely lost: 0 bytes in 0 blocks
indirectly lost: 0 bytes in 0 blocks
possibly lost: 0 bytes in 0 blocks
still reachable: 1,000 bytes in 1 blocks
suppressed: 0 bytes in 0 blocks
What do you think that is? Valgrind tracks allocations by providing other implementations for malloc/free/... .
If the code's running in a space shuttle, you probably want to test that path.
If it's bootstrapping a replicated service, it's likely desirable to crash early if a config file couldn't be opened.
If it's plausible that the file in question is missing, you can absolutely test that code path, without mocking open.
If you want to explicitly handle different reasons for why opening a file failed differently, by all means, stress all of that in your tests. But if all you have is a happy path and an unhappy path, where your code doesn't care why opening a file failed, all you need to test is the case where the file is present, and one where it is not.
Modifying the file system's implementation would be. Including a valid_testdata.txt and an invalid_testdata.txt file in your test's directory, however, is not 'modifying the file system', any more than declaring a test input variable is 'mocking memory access'.
> don't want my daemons or user-facing applications to just crash, when a file is missing
If the file is important, it's the best kind of thing you can do when implementing a non-user-facing service. The last thing you want to do is to silently and incorrectly serve traffic because you are missing configuration.
You want to crash quickly and let whatever monitoring system you have in place escalate the problem in an application-agnostic manner.
Mostly it wraps system calls and library calls. Wrapping means that it does some checking or recording before and maybe after the call. Very occasionally it needs to modify the arguments to the call. The rest of the time it passes the arguments on to the kernel or libc/libpthread/C++ lib.
There are also functions and syscalls that it needs to replace. That needs to be a fully functional replacement, not just looking the same as in mocking.
I don’t have any exact figures. The number of syscalls varies quite a lot by platform and on most platforms there are many obsolete syscalls that are not implemented. At a rough guess, I’d say there are something like 300 syscalls and 100 lib calls that are handled of which 3/4 are wrapped and 1/4 are replaced.
Sorry that wasn't my intention. You are a Valgrind developer? Thanks, it's a good project.
It seems like I have a different understanding of mocking than other people in the thread and it shows. My understanding was, that Valgrind provides function replacements via dynamic linking, that then call into the real libc. I would call that mocking, but YMMV.