Re: [computer-go] taking advantage of board symmetries in neuralnetworks

[Thomas Johnson <thomas@xxxxxxxxxxxxxxxxxxxxx>]

It sounds like your modification of the backprop algorithm might be the
culprit. I don't completely understand what you mean by taking the
average error of similar nodes, but if you're doing something like
averaging the error of your network at each of the four corners, that's
going to mess stuff up. 

One alternative way to take advantage of board symmetries might be to
use each board position to generate four training cases per board
position (one for each board rotation). This doesn't use the inherent
structure of the network per se, but it should help the network learn
more rapidly from fewer board positions. 

Tom

On Wed, 2003-12-17 at 13:17, Frank Steinmann wrote:
> I'm having some experience with a neural network at the moment. My network
> (for 9x9 go) has three fully connected 9x9 layers (so each node of the
> hidden layer has 81 inputs and 81 outputs). For my experiments, I'm using
> random-players, which make any legal moves (except for moving into their own
> eyes) as long as possible.
> 
> First thing I'm trying to learn is the board evaluation at the end of a
> game, black territory (only eyes in this case) gets value 1, white territory
> gets value -1, occupied intersections get value 0. Of course that's easy to
> do without a neural network, but I want to keep it simple at the beginning.
> 
> In my first experiment, I initialized the weights with random values and let
> the network learn from a few hundred end positions, using the back
> propagation algorithm. After that, the nework performed quite well
> (performance could have been better but there was a tendency towards the
> correct results).
> 
> After that I thought about taking advantage of the board symmetries, so I
> initialized the weights of similar connections (according to the symmetries)
> with equal values. To maintain the equality of the weights during the
> learning process it was necessary to modify the back propagation algorithm.
> I took the error of similar nodes, computed the average error, and replaced
> the errors of these nodes by the average error. This caused similar
> connections to keep equal weights.
> The results after learning from a few hundred bad were bad. The network
> didn't seem to learn at all. Maybe the average error was not convenient, so
> I tried the same thing using the largest error of similar nodes instead of
> the average error, but results weren't better.
> 
> So my question is how to take advantage of the board symmetries here?
> Apparently my approach didn't lead to the desired success. Does anyone have
> any experience with that? I'm sure you have... ;)
> 
> Frank
> 
> _______________________________________________
> computer-go mailing list
> computer-go@xxxxxxxxxxxxxxxxx
> http://computer-go.org/mailman/listinfo/computer-go
-- 
Thomas Johnson <thomas@xxxxxxxxxxxxxxxxx>
_______________________________________________
computer-go mailing list
computer-go@xxxxxxxxxxxxxxxxx
http://computer-go.org/mailman/listinfo/computer-go