Re: [computer-go] taking advantage of board symmetries in neuralnetworks

["Nicol N. Schraudolph" <compgo@xxxxxxxxxxxxxxx>]

> So my question is how to take advantage of the board symmetries here?
> Apparently my approach didn't lead to the desired success. Does anyone have
> any experience with that? I'm sure you have... ;)

Our experience (http://n.schraudolph.org/pubs/gochap.pdf) has been
that while board symmetries can be used to get a better evaluation
(by averaging the evaluation for all reflected and rotated positions)
from a trained neural net, those symmetries can actually impede
neural net learning:

Even to learn a symmetric function, a neural net's hidden units may need
non-symmetric receptive fields to do their job.  Constraining the net
by equalizing corresponding weights without crippling it is therefore
tricky - you basically have to replace each hidden unit by a group
of eight, one responsible for each reflection/rotation, and equalize
the corresponding weights among the units in each group, so that each
group as a whole is symmetric, while individual hidden units are not.
This works but (in our experience) is not worth the complications.

Another idea is to symmetrize the training data by presenting each
pattern under all reflections and rotations.  This might lead to a better
evaluation function eventually, but primarily it slows down learning
quite a bit.  We didn't look into it very much, but our hypothesis was
that the symmetric data makes it much harder for the hidden units to
break symmetry, something which they have to do in the learning process.

Overall we felt that dealing with reflections/rotations was not worth
the effort, at least not at the level we were at.  (Disclaimer: all this
was 10 years ago!)  We ended up using color reversal (negation, designed
into the net) because it was easy to do, translation invariance because
it helped *a lot*, and reflection/rotation (only on evaluation, not when
learning) when we had enough cycles to spare (i.e., rarely :-).

Hope this helps,

- nic

-- 
    Dr. Nicol N. Schraudolph                 http://n.schraudolph.org/
    Steinwiesstr. 32                         mobile:  +41-76-585-3877
    CH-8032 Zurich, Switzerland                 tel:      -1-251-3661

_______________________________________________
computer-go mailing list
computer-go@xxxxxxxxxxxxxxxxx
http://computer-go.org/mailman/listinfo/computer-go