I've only just started studying neural networks, so take this with a grain of salt. I'm stumbling around in the normal way of beginning complex subjects, avoiding things like unintuitive algebra and heading toward intuitive visualizations of ideas wherever possible til I get a sense of the field, and I come across this image where you can see the neural network operating on the task of separating a red and a blue spiral from each other. This illustration comes from an article Neural Networks, Manifolds, and Topology
As I'm watching it, I suddenly get the next little piece of information I'm trying to understand. It took a few views but finally I have an inner visualization on how a neural network iterates through what appears to be random guesses toward a destination, and eventually arrives. (The next illustration in the article shows a failed attempt to do the same thing, and points out how a deeper net would do this more efficiently and wouldn't fail, and it all starts to come together.)
Then it occurs to me, according to this visualization, what is happening is quite different from what happens in my own brain when given the same task. The neural net is walking across the outer surface of the assigned task, whereas what happens in my brain, at least as far as I can tell, is more of a quantum leap.
I'm sure my brain's analysis/perception happens so fast I can't break it into small steps like is shown in this image, but I'm now wondering if neural nets operate in this surface-only approach always or just in this particular kind of example. I'll know more as I study further.
[Edit] Thinking about this some more, it seems that I'm bringing a 3-dimensional process to bear in my brain, whereas this example, and many other neural net examples thus far, is two-dimensional. If so, as we give neural nets 3D data, do we step out of positivism? I still think not. Not yet...