This is a good summary. Now, here's what you need to know to have it work:
* initialize weights to small random values (otherwise it will quickly get stuck in a non-desirable local optimum)
* you need to decrease the learning rate of the stochastic descent according to a 1/(n+n0) or similar schedule, otherwise there's a risk of the whole thing swinging back and forth
* it often helps to normalize the input values so that they have a standard deviation of 1 and an expectation of zero. Or at least it's not good to have input values differ too much in scale.
* if you're training something that should give yes/no answers, you don't really want to minimize the mean square error, but a logistic loss.
Some of these parts are usually built into your NN toolkit, so you don't have to worry about them. Or they're not (e.g., choosing a good learning rate), in which case you're just screwed if you don't know what happens and why.
But you're right, mathematically ANNs are quite simple (and, as one has found out, don't really do similar things to what actual neurons do).
* initialize weights to small random values (otherwise it will quickly get stuck in a non-desirable local optimum)
* you need to decrease the learning rate of the stochastic descent according to a 1/(n+n0) or similar schedule, otherwise there's a risk of the whole thing swinging back and forth
* it often helps to normalize the input values so that they have a standard deviation of 1 and an expectation of zero. Or at least it's not good to have input values differ too much in scale.
* if you're training something that should give yes/no answers, you don't really want to minimize the mean square error, but a logistic loss.
Some of these parts are usually built into your NN toolkit, so you don't have to worry about them. Or they're not (e.g., choosing a good learning rate), in which case you're just screwed if you don't know what happens and why.
But you're right, mathematically ANNs are quite simple (and, as one has found out, don't really do similar things to what actual neurons do).