Re: computer-go: Grading Go Problems

[David Fotland <fotland@xxxxxxxxxxxxxxx>]

We could start the grading by looking at which problems are solved by 
existing programs.  If I get access to the
problem set, I can tell you which problems Many Faces gets correct.   It 
will take about a couple of days to run through
such a big problem set.  Many Faces does about 1000 life and death problems 
an hour.

David

At 10:35 AM 7/30/01 -0700, you wrote:


>  Ok, we know that we want them to be graded. We would really like them to 
> be graded in terms of computer go knowledge needed to complete them, such 
> that as you add knowledge to your program, you can complete the next set 
> of problems. This is subtly different then human go knowledge...
>
>   That is, the problems should be broken down into: Elementary School, 
> High School, College, Graduate School, and Nobel Prize. :-)
>
>   Here is my initial pass of some starting level:
>
>   1. The first level should be the basic eye shapes in three versions: 
> corner, edge, center. If you have a program that recognizes eye shapes 
> already perhaps you could tweak it to pull relevant problems out of the 
> database.
>
>   There's no reason why I can't include other problems that people want 
> to contribute. If you already have a set of files for the basic eye 
> shapes, its probably worth contributing.
>
>  2. The next level is "making good eye shape". These would be a set of 
> problems that are "one move away" from a living eye shape. If your 
> program can already recognize eye shapes, perhaps you could tweak it to 
> recognize these while its trying to solve them.
>
>  We can probably generalize this to 2, 3 moves away to fill in some levels?
>
> 3. The level after this would be tesuji. Not sure how we would recognize 
> these. Perhaps if you have a set of tesuji patterns you could report ones 
> that fired during the solution of the problem, and then we could sort the 
> problems such that we get a distribution of the tesuji.
>
> 4. Connecting? Problems that involve connecting or breaking connections?
>
> 5. Ko. Problems that involve Ko.
>
> 6. Ko Threats. Go Tools distinguishes between life/death and life assuming 
> some number of successful ko threats. In my experience most go programs 
> aren't this smart, but it seems like someday they will have to become that 
> smart!
>
> Grades past that:  Problems that involve more types of go knowledge 
> together, such that a program has to choose its moves more appropriately
>
> Next to Final grade:  Really tough, "interesting" problems.
>
> Final grade: "weird" problems. Problems that programs tend to have great 
> difficulty with.
>
> Pierce