RE: An AI program that doesn't learn

[Mika Kojo <mkojo@xxxxxx>]

Peter.Smith@xxxxxxxxxxxxxxxxx writes:

> If the program is tuning itself against a set of problems then again it's
> not interesting for the program to keep history information for a board
> position (it won't have any when it is being trained.)

Yes, in supervised learning it is not necessary to store the previous
states. Many reinforcement methods work with history information.

> Where this would apply is if the program is itself learning by playing
> matches against humans or better still other programs.  I have the gut
> feeling this approach is simply going to take too long to be viable. 

The approaches with temporal difference learning use previous
positions, and they prove your point --- it takes a long time to learn
to play Go.

I believe that TD methods work nicely for recorded expert games, and
might perform better than the usual "strict" supervised learning in
many situations. In the TD approach storing the previous positions is
often necessary, and knowing them is always necessary.

However, there still is a place for supervised learning as TD methods
demand some care with the input data.


Mika Kojo