Re: Fw: Re: [computer-go] Statistical Significance (was: SlugGo v.s.Many Faces, new...

[Harry Wang <foreversoldier@xxxxxxxxx>]

well, I agree that there is only one outcome - porgram
either win or loses. However, when I look at the
strength of two programs - such as SlugGo and GnuGO,
variance is a good indicator to show strength of a Go
program. 

given 80 games played between two programs, if I have
one program winning 70% of the time with a small
variance. Then I am very skeptical about the result.
Then I play more matches using two programs - say
another 300 games. Usually the winning percentage gets
smaller when more games are played. 

With larger variance, my personal confidence level is
higher that one program is stronger than another. 

Harry Wang


--- petrip@xxxxxxxxxxxxxxxxx wrote:


---------------------------------
body{font:12px
Arial;margin:3px;overflow-y:auto;overflow-x:auto}p{margin:0px;}blockquote,
ol, ul{margin-top:0px;margin-bottom:0px;}
Since this is a binary process - Program either wins
or loses - we do not need to measure variance. It is
binomial distribution with well known properties.
Measuring that would be bit pointless as result is
known beforehand

 

Petri Pitkänen






---------------------------------


> From:: Compgo123@xxxxxxxxxxxxxxxxx
> To: computer-go@xxxxxxxxxxxxxxxxx
> Subject:: Re: [computer-go] Statistical Significance
(was: SlugGo v.s. Many Faces, new...
> Date: 09/08/2004


The fluctuations of the test results is related
(proportional?) to the variance of the test value.
(Also the distribution function. A Gaussian? A
Poisson?) The confident level can be calculated from
the value of variance. Thus we need to know what's the
variance of the outcome of games between two Go
programs. For difference handicaps the values of the
variance could be different. Is this variance
independent of which these two programs are? To
approximate one can determine the values of variance
between two GnuGo programs for different handicaps.
Also the variance between GnuGo and Manyfaces of Go
for different handicaps. Since David  Doshay's program
is the offshoot of GnuGo, these variance values could
be good approximations. This way one can obtain the
values of variance on any PC through a large number of
games. Save the more valuable CPU time for the 72-cpu
cluster.

Daniel Liu  


Heart disease is Britains biggest killer. Join the
British Heart Foundations Big Red Fightback:
bhf.org.uk/fightback
> _______________________________________________
> computer-go mailing list
> computer-go@xxxxxxxxxxxxxxxxx
>
http://www.computer-go.org/mailman/listinfo/computer-go/



		
_______________________________
Do you Yahoo!?
Win 1 of 4,000 free domain names from Yahoo! Enter now.
http://promotions.yahoo.com/goldrush
_______________________________________________
computer-go mailing list
computer-go@xxxxxxxxxxxxxxxxx
http://www.computer-go.org/mailman/listinfo/computer-go/