Re: [computer-go] Time constraints for computer Go

[Peter Schay <peteschay@xxxxxxxxx>]

I agree -- progress against humans is the real
challenge in computer go.

Computer tournaments are interesting and valuable for
many reasons, but "progress" ought to be measured
against humans.

It's been a while since I read the paper, but IIRC,
didn't Schaeffer report diminishing returns for search
in checkers? 
(http://citeseer.ist.psu.edu/schaeffer93reexamination.html)
?  Since go seems to reward human strengths (such as
knowledge and creativity) even more than checkers,
there's a chance we might see rapidly diminishing or
even disappearing returns for brute force search in go
against humans.  (Computer vs. computer is a different
story).

Here's a possible experiment to chart the returns
versus humans of brute-force as a function of time (or
as a function of search ply) -- what do you think,
does the following make any sense?

First, on the small scale: Enlist some volunteers. 
Give the humans an hour in every match and give the
computer more time for each match in the series.  
Plot the game score results as a function of computer
think-time.  Whether the computer gets less time or
more time than the human does not matter; just that
the human think-time stays constant and the computer
think-time is varied in the experiment.

Now on a longer-than-usual scale: Play a kind of
"correspondence" go.  Give the human a half a day per
move, and give the computer or cluster a smaller
amount, say 30 minutes.  Report the computer move to
the human at noon every day, and require a reply by
midnight.  Play a number of matches with different
amounts of time per move for the computer.  This
experiment could finish within a few months and would
not be unreasonable for the people playing.  Also the
same computer resources could be used to play against
multiple people during each day, depending on the
chosen computer time limits of course.


Has anything like this been tried in chess?  What were
the results?

Pete

--- Mark Boon <tesujisoftware@xxxxxxxxxxxxxxxxx> wrote:

> The discussion about time-constraints seems to be
> divided between the
> practical and the academic. Both sides have valid
> arguments.
> Personally I think the main objective of computer-Go
> is to make
> software that we can match against a human. There IS
> value in writing
> software purely from an academic point of view, but
> I think to most of
> us the real interest is in having software compete
> with humans to see
> to what respect the software can compete in
> 'intelligence' with the
> human brain.
> 
> Although a one hour time-limit is somewhat
> arbitrary, I think the
> criteria that a human must be willing/able to play
> against the
> software is not. A one hour time-limit seems like a
> fair approximation
> of that.
> 
> At the moment SlugGo is the only software that
> scales well. But I
> think this is a temporary situation, which is partly
> actually caused
> by the constraints imposed in the past by the
> computer power then
> available. Making a program scalable was less of a
> priority than
> adding other constructs that would improve play
> within the given
> constraints. As soon as there are more programs that
> are scalable, it
> will become obvious that the question is no longer
> who can make the
> best playing program, but who can make the best
> playing program within
> a certain time-constraint. To link this
> time-constraint to the
> time-limit a human player is comfortable with seems
> only natural.
> 
> The reason I think SlugGo is an important
> development in computer-Go
> is twofold. Firstly it shows it may be around time
> to look at scalable
> Go programs as it may provide an easier way to
> improve the level of
> play through scalability than adding non-scalable
> constructs.
> Secondly, the gain in strength SlugGo shows is much
> more than the
> progress of computer-Go over the last decade. It is
> quite possible
> that a decade from now computers will be powerful
> enough to enable
> SlugGo to play against a human, running on a normal
> PC.
> 
> Most Go programmers will have had good ideas for
> their program only to
> find that although it improved play, it made it too
> slow. Instead of
> being discouraged by this and rejecting the idea, I
> think most of
> those times we have looked for ways to optimize or
> modify the program
> until it would play fast enough and still make use
> of the new
> idea/insight. This is the stage that SlugGo is now
> in. Or we can wait
> five to ten years for hardware to catch up. We'll
> see then if other
> programs will have improved just as much in the
> meantime or not.
> 
> Bottom line (in my opinion): the best Go program is
> the one that plays
> best against humans, at least as long as the best
> humans are stronger
> than the best software. SlugGo is not there yet as
> it doesn't seem to
> play at a pace that is healthy to humans, but it
> shows very good
> promise. I fully understand David's wish to test his
> program/ideas
> against more programs and his progress is
> interesting enough that we
> should look for ways to make that possible. But I
> don't see why we
> would change tournament conditions for his program.
> 
>     Mark
> _______________________________________________
> computer-go mailing list
> computer-go@xxxxxxxxxxxxxxxxx
>
http://www.computer-go.org/mailman/listinfo/computer-go/
> 

_______________________________________________
computer-go mailing list
computer-go@xxxxxxxxxxxxxxxxx
http://www.computer-go.org/mailman/listinfo/computer-go/