Neural Network

Neural networks are not new; they have been around since 1943. But although the neural calculus had been worked out, no automated method of setting the network's connection strengths was known.

To better appreciate the challenges of multivariate function estimation, a number of classical approaches to univariate function estimation are explored. These approaches broadly fall into three areas.

- Nonlinear Regression
- Empirical Model Building
- Nonparametric Regression

Univariate methods rapidly become more difficult as the number of inputs increase. Because of the difficulty specifying the functional forms of high-dimensional equations, it is uncommon to use parametric nonlinear regression for more than one or possibly two inputs. And nonparametric regression methods, such as smoothing splines, suffer because of the relative sparseness of data in higher dimensions.

Neural networks are also flexible multivariate function estimators. Unlike classical methods, however, neural networks perform well in sparse, high-dimensional spaces.

- Potts, WJE. Neural Network Modeling –