Re: [computer-go] future KGS Computer Go Tournaments - two sections?

[John Tromp <John.Tromp@xxxxxx>]

some people already made similar suggestions about a similarity test,
but here's some thoughts on making it a little more concrete:

Someone could write a pseudo-random position generator,
that creates a set of say, 100,
19x19 positions with various numbers of stones on the board in
patterns that more or less resemble human play. The positions
should have sufficient "quiescence" so that many moves are reasonable.

Then if we fix some time controls and let two programs find the best move
for each position, we find their level of similarity.

If program BlackBox and GnuGo produce the same move in 87 of the 100 positions,
we can declare them 87% similar. This would indicate that BlackBox is a very
close derivative of GnuGo.

We might imagine a computer tournament in which any two entries that are
50% or more similar must be "resolved" so that one of them is excluded.
Each such similarity test will generate a new batch of positions, so that
one cannot rig the test by trying to recognize the positions. The number
of positions in the test should also be high enough that the standard
deviation is within reasonable limits.

If someone wants to enter a GnuGo derivative in such a tournament, they would
have to ensure low similarity by for instance, randomly choosing among the
top-3 moves. This would however make the program rather weak as it will miss
a lot of "forced" moves.

So instead it could randomly choose among all the
moves that are within so many points of the optimal move.
I suspect that such programs will also be noticeably weaker than GnuGo.

In any case, it might be interesting to develop such a pseudo-random
position generator and see what levels of similarity exist between various
well known programs.

regards,
-John
_______________________________________________
computer-go mailing list
computer-go@xxxxxxxxxxxxxxxxx
http://www.computer-go.org/mailman/listinfo/computer-go/