[computer-go] SlugGo vs Many Faces, newest data

[David G Doshay <ddoshay@xxxxxxx>]

In our first set against Many Faces of Go, where we played White with 
no handicap and no komi and the results were:
> 	Black wins 5 (by: 33, 15, 75, 4, 21)
> 	White wins 20 (by: 126, 59, 67, 73, 40, 17, 25, 76, 58, 21, 25, 125, 
> 24, 2, 42, 54, 41, 1, 32, 53)
After I posted the above set Doug Ridgway suggested we start giving 
Many Faces a handicap, so we ran some 2, 3, and 4 stone games.

Through 28 August we generated the following results. SlugGo is always 
white against Many Faces of Go playing black, no komi.

Handicap = 2
	42 games
	Black won 11 (by: 11, 3, 45, 66, 31, 36, 3, 17, 27, 7, 3)
	White won 31( by: 33, 31, 40, 2, 25, 23, 9, 56, 57, 17, 14, 21, 65, 
37, 77, 16, 91, 51, 78, 15, 4, 2, 74, 6, 35, 23, 30, 18, 9, 31, 51)

Handicap = 3
	6 games
	Black won 4 (by: 25, 135, 30, 107)
	White won 2(by: 21, 51)

Handicap = 4
	18 games
	Black won 8 (by: 11, 36, 58, 15, 12, 6, 13, 56)
	White won 9 ( by: 11, 104, 79, 49, 20, 72, 51, 24, 10)
	One tie

I sent the above to Doug, to which he replied:
> There's a plot of the data, best
> fit line, and 95% CI of score predictions at
> http://dridgway.com/Go/sluggo_vs_MFG.png .

Doug also suggested we try higher handicap games. We did,
although we did not collect very many.

Newest results since we reported the above to Doug are:

Handicap = 3
25 games
SlugGo wins 13 (by: 41, 20, 25, 50, 15, 23, 8, 61, 49, 66, 17, 9, 11)
MFG wins 12 (by: 39, 4, 14, 17, 74, 136, 73, 52, 61, 88, 2, 29)

Handicap = 4
27 games
SlugGo wins 14 (by: 1, 3, 76, 17, 32, 22, 1, 16, 48, 29, 17, 37, 91, 68)
MFG wins 13 (by: 19, 39, 43, 107, 26, 57, 21, 16, 22, 16, 3, 66, 93)

I found it interesting that Many Faces won 10 of the first 13, and then
SlugGo won 11 of the next 14. This was what I meant in an earlier
email when I wrote about how easy it is to be fooled by small samples.
It looks to me like we would need at least 200 games to know what
is going on at 4 stones. All of these games involved huge groups and
large strings of cut off stones.

Handicap = 5
6 games
SlugGo wins 2 (by: 1, 31)
MFG wins 4 (by: 15, 33, 17, 8)

Handicap = 6
5 games
SlugGo wins 2 (by: 13, 36)
MFG wins 3 (by: 24, 5, 7)

I did not expect to win any 6 stone games, so I was pleased. But as said
earlier, these are very small samples and I do not trust small samples.

Perhaps Doug will be so kind as to add the new data to the other and
send along a pointer to a plot of the full data set.

This will be the last of the data from playing SlugGo against Many Faces
for a while. We are going to concentrate on cleaning up the code and
changing the logic in a few ways. I do not plan to play any more games
with the present code, but will start again fresh with the new version.

Cheers,
David


_______________________________________________
computer-go mailing list
computer-go@xxxxxxxxxxxxxxxxx
http://www.computer-go.org/mailman/listinfo/computer-go/