I've made the jump from physics to neuroscience, so I can talk to the engineers here (I've taken a lot of EE and worked professionally in it too).
The linkages between neurons is somewhat similar to how and RNN looks. But you must remember, there are electrical and chemical elements going on here. It's not just one neuron spiking another. There are many different biochemical processes that modify the behavior of little parts of a neuron, stoichiometrically. And there are many different types of neurons and they all change over time, sometimes drastically so. Most of the goings on is biochem. It's not digital, or even analog. You really need to go down to the field equations at times, finite elements will get you far, but only just so.
RNNs thinking certainly will help you understand better what is going on in a brain, but, like, these things are millions of years old, and optimized just to make more of themselves, not to be understood. It's tough going, and we as a species are only at the very beginning of hundreds of years of study of the brain.
If you'd like to learn more, I can recommend some texts.