RE: [computer-go] Computer Go Test Collection 2.0

["David Fotland" <fotland@xxxxxxxxxxxxxxx>]

For Many Faces I have about 1500 simple life and death problems, and
about 200 fuseki and endgame problems.  The hardest part to test is 
middle game attacking/reducing/surrounding strategy.

David

> -----Original Message-----
> From: computer-go-bounces@xxxxxxxxxxxxxxxxx 
> [mailto:computer-go-bounces@xxxxxxxxxxxxxxxxx] On Behalf Of 
> Gunnar Farnebdck
> Sent: Thursday, June 10, 2004 8:00 PM
> To: computer-go
> Subject: Re: [computer-go] Computer Go Test Collection 2.0 
> 
> 
> Markus wrote:
> > Version 2.0 of the Computer Go Test Collection has been released. 
> > [...] We welcome feedback on the test set: modifications 
> and additions 
> > to the test cases, alternative good (or bad) moves, and 
> comments about 
> > which types of tests are most useful.
> 
> I won't comment too much on this specific collection. I have 
> looked at it quickly but for GNU Go it is somewhat redundant 
> since we already have our own very big test collection. Let's 
> note however that the format used in this collection is 
> exactly the same as GNU Go is using and that the analysis 
> tool in GoGui is interchangeable with the tools in GNU Go.
> 
> If you don't already have a testing framework in place for 
> your engine, I warmly recommend using this one. For GNU Go it 
> has been invaluable and pretty much a prerequisite for the 
> strength increase over the last four years.
> 
> When it comes to which types of tests are *most* useful, my 
> experience is that those can be found in two categories: 1. 
> Systematic and serious errors your own engine insists on 
> doing. 2. Systematic collections of specific types of 
> problems, preferrably
>    in artificial settings without any distractions.
> 
> The reason why I prefer artificial settings over real-game 
> positions in the latter category is that it allows you to 
> concentrate on a specific part of the engine and makes 
> debugging/tuning as efficient as possible. They also tend to 
> be faster to run, which does become an issue once your test 
> suite becomes big. The reason for a systematic collection is 
> that you can get a good idea of what is needed to solve a 
> whole class of problems by analyzing the failures in many 
> more or less closely related situations.
> 
> The main difficulty with category 2 is that it takes a lot of 
> effort to create such collections. It is much easier to 
> identify mistakes in games your engine is playing and adding 
> those to the test suite. In GNU Go the following two test 
> files are (mostly) of category 2:
> 
> ld_owl.tst:
> Life and death tests, mostly basic corner shapes, including a 
> big section with variations on the tripod shape. A drawback 
> of this collection is that it uses GNU Go specific GTP 
> commands to query life and death status of groups, which are 
> not necessarily easy to emulate in other engines.
> 
> seki.tst:
> Corner formations which can be invaded to make seki, plus 
> followup moves. Uses only standardized commands except for a 
> few tests at the end.
> 
> You can find those files in the regression directory of e.g. 
> the GNU Go 3.5.8 distribution.
> 
> I know that there is ongoing work (outside of GNU Go) on a 
> semeai test suite of this kind, which I hope will be released 
> to the public. If somebody knows of other collections of a 
> similar nature, I would be very interested in seeing them.
> 
> /Gunnar
> _______________________________________________
> computer-go mailing list
> computer-go@xxxxxxxxxxxxxxxxx 
> http://www.computer-go.org/mailman/listinfo/computer-go/
> 

_______________________________________________
computer-go mailing list
computer-go@xxxxxxxxxxxxxxxxx
http://www.computer-go.org/mailman/listinfo/computer-go/