I don't know about the "hidden ingredient", but...
When I lived in upstate NY our house was on property that had been a glacial lakebed. The soil was a bit sandy, if anything too well drained, but adding lots of leaf mold -- we had more than enough maple trees -- made it retain water adequately, and things grew well.
When it came time to move, we sold the house to a pair of doctors who were moving up from Texas. One of them was just so enthusiastic about how good the soil was, and about the big piles of partially decomposed leaves we had, compared to the terrible clay soil they had down there. So if nothing else, good soil can help property values.