All this talk of simplicity may mislead one into building something that just about works, and calling it a day. I believe that is a mistaken approach in science and engineering. Instead, there should be a deeper understanding of the limits and constraints on the problem. In my view, the focus should be on the problem and its constraints rather than on the nature of the solution.
An analogy in terms of algorithms would be: Why take the pains to implement an O(nlogn) solution to sorting when you can implement a O(n^2) solution which is far simpler. ie, be satisfied with a feasible solution instead of seeking what could be more optimal. The road to interesting insights may not be linear and incremental. Doing the simplest thing at all times is an incremental greedy approach.