Re: [computer-go] SlugGo v.s. Many Faces update

[Evan Daniel <evanbd@xxxxxxxxx>]

On Thu, 31 Mar 2005 16:24:40 -0800, David G Doshay <ddoshay@xxxxxxxxxxxxxxxxx> wrote:
> We have fixed the problem with our setup that caused Many Faces to
> think it was running out of time ... MF did not like being told it
> always had 999 minutes left to play. We have verified that MF stays at
> level 10 for the entire game, and for all games in a series.
> 
> Our biggest problem is that it takes hundreds of games to know how the
> programs compare. We have runs where one program will win 14 games in a
> row, but overall the ratio is not nearly that one sided. It is very
> difficult to know when the statistics are "stable."
> 
> The biggest change to SlugGo is that it can now branch at levels beyond
> the first ply. We statically define the branching factors before each
> set of games.

It's starting to sound more and more like GoFigure :)  (With the
exceptions that GoFigure isn't clusterable and only plays 9x9, and
searches to the end instead of a fixed depth...)

Have you solved the problems you were having with poor opening play (I
have my own answers to that question, and would be curious to
compare)?  What branch factor settings are you using, and have you
noticed much strength variance?

In GoFigure, I have seen significant strength dependence on the branch
factor settings.  I've tried 2 vs 3 branches at ply 0, none deeper
(the third option didn't help much), 2 branches at each of plies 0 and
1 (oddly appears to be weaker, though my only properly automated
opponent so far has been GNU Go (played at a stronger setting that the
internal copy I'm querying).  I'm a little unsure how much of this is
the obvious artificial effects vs real effects, and it will get some
investigation when I have time (probably ont soon...).  Branching
twice at plies 0-2 seems stronger than either other option (though for
speed reasons has gotten less testing).  Whether it's worth the speed
tradeoff is open to debate :)

> 
> For the branching factors I have been running on our new mini cluster
> for the last many days, of 63 games with SlugGo playing white with a
> 5.5 point komi, SlugGo won 42 and Many Faces won 21. The 2 largest wins
> for SlugGo were by 101.5 and 159.5. The two biggest wins for Many Faces
> were  56.5 and 63.5. The closest game was a 0.5 point win for SlugGo.
> With different branching factors we have seen an overall win/loss ratio
> of 2.75 for 88 games.
> 
> The only thing that is very clear is that the variance is BIG. The
> space is so very large and it takes a very long time to collect 100's
> of games. We have seen games take anywhere from 1 hour 40 minutes to 3
> hours 45 minutes (total for both sides) on the G4 Mini-mac cluster.
> 
> We will start playing with different handicaps soon.
> 
> And we have started adjusting the parameters inside SlugGo. I am trying
> to keep that to a minimum just to avoid tuning specially against Many
> Faces ... but so far MF is our only automated opponent.

Sounds good, I can't wait to hear about more results.

Evan
_______________________________________________
computer-go mailing list
computer-go@xxxxxxxxxxxxxxxxx
http://www.computer-go.org/mailman/listinfo/computer-go/