Re: ANN

[Scott Dossey <sdossey@xxxxxxxx>]

On Wed, 29 Apr 1998, Weimin Xiao wrote:

> 
> Jens,
> 
> I am looking at ANN issues these days. One thing bothering me is that:
.
.
.
. Much text deleted for brevity's sake.
.
.
.

> 
> The conclusion: one and only one hidden layer in an ANN is needed.
> 
> What do you think?
> 

Yes.  Indeed, it has been proven that this is true.  A two layer ANN can
do linearly separable problems and a three layer ANN can do *anything*.
There is a nice proof of this that I won't go into that somewhat is
related to the fact that any logical function is possible to do with only
two layers of logical gates in a circuit.  Albeit--those sometimes have to
be very wide and use and a impractical number of gates.

Similarly, while a three layer ANN can go *anything* doesn't mean four
layers aren't useful.  The more nodes you add to an ANN the less like it
is to get stuck in a local minimum.  Similarly, adding layers helps the
ANN to break up the problem into "baby steps" which can cause it to
converge more quickly and helps to avoid local minimums.


-Scott Dossey