Q:

What in neurons and their connections changes during the process of learning?

I'm not sure if this question belongs more in physics or biology (or maybe even computer science)... but biology seemed to fit more.
What changes in the state of our brains when we learn things?  Because I looked on the internet and I learned about artificial neural networks, and every resource I've found teaches of neural networks that have weights that are trained/evolved and then are static at runtime; you simply train the weights and then once you use them in simulations, they never change.
Purely feedforward neural networks that rely on this can't react to new situations differently if they've experienced it before.  They react exactly the same each time.
I suppose that theoretically, a recurrent neural network that was big enough could really learn, but in practice, these have been used as purely memory slots for an existing method defined by the weights, not as storage of new methods of doing things.
So my question is, physically, chemically, biologically, what changes in the neurons and the connections between them when we learn things?  I don't think we really understand how it comes together on a grand scale yet, but I'm pretty sure we've figured out that much.  I want to learn how to model a simplified version of it mathematically/programmatically.

A:

What changes in the process of learning:

The connections (the way one neuron is connected to another). New synapses can form or dissolve in the process of learning. The glial cells such as astrocytes and microglia can facilitate this process.
Strength of connections: Existing connections can be made weaker or stronger in the process of triggering the same circuit. This happens by up/down regulation of the ion channels at the synaptic junctions.

This process is not fully understood and is an active area of research. You can find some mathematical models too for learning processes such as Long Term Potentiation etc.

