RE: [computer-go] future KGS Computer Go Tournaments - two sections?

["Lucas, Simon M" <sml@xxxxxxxxxxx>]

 This is a really interesting and
 thorny issue - I don't have any
 good answers, but just a couple
 of observations.

 One point I'd like to make is that
 it's not very sensible to measure the
 difference in programs in terms of the man-hours
 spent developing them - there is no simple
 relationship between time effort
 and 'distinctness' or playing ability.  
 Sometimes a relatively minor code change can
 make a big difference to performance.

 I'm not too familiar with these programs
 in particular, but in general, the set of
 tunable parameters may be a major part the
 identity of a game-playing program, so it
 may be fair to see these as different entries.

 That's all complex enough when everyone plays
 fair - but one other possibility is for
 a set of similar programs in a tournament to
 collude by all losing to their selected 'winner'
 (a bit like sometimes happens in Formula One Racing).

   Simon Lucas


 

--------------------------------------------------
Dr. Simon Lucas
Department of Computer Science
University of Essex
Colchester CO4 3SQ
United Kingdom
Email:  sml@xxxxxxxxxxxxxxxxx

http://cswww.essex.ac.uk/staff/lucas/lucas.htm
--------------------------------------------------

 

-----Original Message-----
From: computer-go-bounces@xxxxxxxxxxxxxxxxx
[mailto:computer-go-bounces@xxxxxxxxxxxxxxxxx] On Behalf Of David Fotland
Sent: 11 May 2005 06:44
To: 'computer-go'
Subject: RE: [computer-go] future KGS Computer Go Tournaments - two
sections?



Let me try to explain with a though experiment.  Let's say, without any
data, that Many Faces of Go can beat Gnugo 70% of the time in 100's of
test games.  Both programs can randomize openings, and Many Faces
learns, so it won't lose the same way twice.  I'd be willing to enter a
tournament against one copy of Gnugo, expecting a 70% chance of winning.
I would not be willing to enter a tournament against 10 copies of gnugo.
It is almost certain that I would not win this tournament.

I don't understand how anyone can think that this is fair.

Now let's suppose that instead of 10 copies of Gnugo, the tournament has
10 programs that are minor derivatives of gnugo with similar playing
strength. I think it's obvious that this is equally unfair.

It's just as unfair to allow two gnugo versions rather than 10.

For those who would argue that SlugGo is not a minor deriavative, I
would ask them to compare the total person hours of work put into Gnugo
and SlugGo.  I would be surprised if SlugGo has even 5% of the total.

Many Faces has hundreds of tunable parameters, and I don't know which
set plays best against Gnugo.  Would it be fair  for me to enter 10
copies of Many Faces with different parameter values against one copy of
Gnugo?

Marco, if you were in a race against someone, would it be fair to let
him race twice, on different days, and use his best time against your
single time?  How about if he was allowed to train between the races, so
it wouldn't be the "same" person?

David

> 
> I still don't understand why having both SlugGo and GNU Go in a
> tournament would be unfair and to whom? Why this fix on derivative  
> work? This is how progress is made!
> 
> marco
> 
> 
> _______________________________________________
> computer-go mailing list
> computer-go@xxxxxxxxxxxxxxxxx
> http://www.computer-go.org/mailman/listinfo/computer-go/
> 


_______________________________________________
computer-go mailing list
computer-go@xxxxxxxxxxxxxxxxx
http://www.computer-go.org/mailman/listinfo/computer-go/
_______________________________________________
computer-go mailing list
computer-go@xxxxxxxxxxxxxxxxx
http://www.computer-go.org/mailman/listinfo/computer-go/