computer-go: god rank statistic

[Yannick Delbecque <yannick_delbecque@xxxxxxxxxxx>]

Since god rank was mentionned a many times recently, this story might be
of some interest: a math teacher and go player I know used IGS go server
statistics to
estimate god rank: he first mesured the standard deviation on the score
difference for each rank, and found out that it was fellowing a simple
decresing law in function of rank  (*) - ie as players gets better,
their score are more stable. Then he made the following argument: if that
god player is a perfect go player and that law applies also to him, then
when he
play go with himself the score will be always the same. Using the
statistics, the teacher extrapolated the curve (I thing the law was a
simple linear one) to find out at what rank the standard deviation is
null. That rank - as far as I remember - was IGS 13d. 

The part of that story that is the more interesting to computer go is the
the fact that since a program or anything that can play go that get
better is expected to do less important errors and do them less often,
so a law similar to (*)  should be observed if we do try to calculate god
rank with the statistics we get considering a given bunch of go played
(IGS or a bunch of training programs or anything!). We can give the "god
rank" mesure more sense if we assume that each bunch has a random player
(a player who play randomly) to which we compare "god rank" (eg: in that
given bunch, the estimated god player can give a random player 60
stones, in another, 80) and mesure rank (or "stones") in the same way
in each bunch. 

Now come standard statistical considerations: how good are estimates of
"the absolute god rank", the god rank obtained by considering the "every
possible go player" bunch 'B' obtained by a given "subbunch"? 
Statistics teaches us that we must take a random subbunch of B (of a
given size, supposed large enough to minimize the error probability up
to taste...) to do estimates that make sense... How "random" is IGS, or
any bunch of human players? It is possible to argue that these bunches
are very biased (for exemple because humans have always missed some very
important fact about go, so they all play very poorly when compared to
the B god. The argument given in a previous mail, "human are not good at
moving things around" may point out such a bias in chess.).

And how do that absolute god rank realte the the ommicient player (O)
rank (a player who know the game tree completely and can use that
knowlege to take optimal decisions). The absolute god rank, as I defined
it, is a satistic, and the O rank is the rank of a specific player, so a
priori they might not be the same! But they must be, since when we
compute the B rank, we take O into account, and O is going to be the top
ranked player and the standand deviation at O rank will be zero (O
playing with himself), so the standard deviation as a function of rank
will cross zero a the O rank, thus proving that B rank = O rank.  

And a last comment concerning the discussion about the
definition of a difficult game: how about mesuring the dificulty of a
game by it's absolute god rank? Of course, the rank will be mesured the
number of move the second player can pass and still get an even game
with a random player... For chess, for exemple, we cannot apply directly
the "god rank" method used for go, for there is no score in chess. We
can use instead of a single game many game with the same opponents and
use the resulting score difference like in the case of go. The same
phenomenon is to be expected: better players playing such tornements
will have scores that do vary less.

Y