ANN

["Weimin Xiao" <ranxiao@xxxxxxxxxxxx>]

Jens,

I am looking at ANN issues these days. One thing bothering me is that:

If I am approaching an existing curve (or should we say, fit a curve), I
might use a complete set of polynomials to do it. For example, if I have
a three-variable problem, then I would start my function as

    f(x1,x2,x3) = a0 + a1*x1 + a2*x2 + a3*x3 +
                               a11*x1*x1 + a12*x1*x2 + a13*x1*x3 +
                               a21*x2*x1 + a22*x2*x2 + a23*x2*x3 +
                               a31*x3*x1 + a32*x3*x2 + a33*x3*x3 + ...

If I am using ANN to do the above, I am using polynomial discriminant
functions. I can add more and more terms to get better results. The draw
back of this is if the order of polynomials become higher, then the
number of parameters (a...) increased very quickly, and referencing
those parameters becomes difficulty.

If I am using layered ANN, say one-hidden layer

                     x1      x2       x3
                        y1       y2      ...
                             z1

the output value z1 is

      z1 = w1*y1 + w2*y2 + ...
          = w1*exp(w11*x1 + w21*x2 + w31*x3) +
             w2*exp(w12*x1 + w22*x2 + w32*x3) + ...

While more hidden nodes added, I am expanding approaching function
f(x1,x2,x3) using exponential function set. So, eventually, a polynomial
discriminant function set will give me the same results as an ANN with
one hidden layer, and they both are complete which also means we need
and only need one hidden layer to fit any continuous function, extra
hidden layer would not help. Also, if a curve is simple, say x*x, we
will not get very good results from a layered ANN for just few hidden
nodes.

The conclusion: one and only one hidden layer in an ANN is needed.

What do you think?

Weimin

-----Original Message-----
From: Jens Yllman <jens.yllman@xxxxxxxxxxxxxxxxx>
To: Weimin Xiao <ranxiao@xxxxxxxxxxxxxxxxx>
Date: Wednesday, April 29, 1998 2:37 PM
Subject: Re: Learn Alternative Moves


... It is the ANN part I'm most interested in discussing...

Jens Yllman