Re: [computer-go] Third KGS tournament: game-end protocol

["William M. Shubert" <wms@xxxxxxxxxx>]

On Tue, 2005-06-07 at 22:03 +0200, Gunnar FarnebXck wrote:
> Nick wrote:
> > The recent third KGS Computer Go Tournament did not see any bot resume 
> > play to establish its claims about status.  What did happen, in four of 
> > the games, was that two bots disagreed about the status of at least one 
> > group, and rather than resume play, they alternately and repeatedly 
> > asserted their claim by marking the disputed group(s) as dead and as 
> > alive.  In the least clear such case, the disputed group was unsettled: 
> > it had one eye, and could make, or be denied, a second eye.  In the 
> > clearest such case, the disputed group had 26 foolproof eyes.  Six of 
> > the eight bots in the Formal division became involved in these 
> > arguments, and one risked missing its game in the next round, preferring 
> > to continue to argue.
> 
> As far as I can see, the less than graceful behaviour in the case of a
> dispute is not the bots' fault but a misfeature in kgsGtp and/or the
> server. Since there apparently never were any cleanup moves generated,
> kgsGtp had no reason to ask the bots about the status more than once
> (and possibly it didn't) and when they didn't agree (possibly because
> one of the bots didn't implement final_status_list) and didn't manage
> to resume, there really was no good reason for the server to start
> blinking the stones indefinitely. It would have made a lot more sense
> for the server to interrupt the game somehow and notify the tournament
> organizer.
> 
> /Gunnar

I am probably at least partly responsible for Nick's request earlier
that all programs in a tournament either play until all dead stones are
removed or implement the "kgs-genmove_cleanup" command. Basically, when
I said it was up to Nick to decide whether or not to require this or
not, I didn't realize some of the problems caused by *NOT* requiring
this. This relates to your comment, Gunnar, so I'll discuss it here.

There is a weakness in the kgsGtp program. When it disagrees with its
opponent about the final score of the game, it will not budge until
either the opponent resumes play, or agrees with it. This means that
when there is a score disagreement between two computer programs, they
will sit and argue until an admin kills the game.

Fixing this is harder than it sounds. Once the TD (Nick) has set the
winner in the tournament system, there is no point in continuing the
argument - but at the moment, the tournament system is a separate
application, so when (for tournament purposes) a game has been marked as
B+ or W+, the actual KGS server doesn't know or care, it still leaves
the game there unscored. So to solve this by letting a human mark the
game win/loss in the tournament system would mean that I would have to
add a communications channel from igobot (the TD program) to the kgsGtp
robots, saying when a game is scored. This would probably have to go
through the server then be relayed out to the players, meaning extra
protocal commands added. That of course would also have to be thoroughly
tested to make sure it didn't cause trouble in human tournaments.
Overally, not a 5 minute fix - probably a few days work once you add the
testing.

Meanwhile, requiring tournament programs to either play until dead
stones are removed or implement "kgs-genmove_cleanup" *IS* a 5 minute
fix. Just a few lines of code: At the end of all tournament games, check
whether "kgs-genmove_cleanup" is supported. If not, then claim all
stones on the board are alive. If it is, and there is disagreement, then
resume play. End of story.

So after the problems with the programs disagreeing on score, then
sitting and arguing instead of continuing on with the tournament, I told
Nick that he is welcome to keep running tournaments this way, but that I
didn't plan on fixing the "sit and argue" bug, so he would need to make
sure an admin was on hand to kill these problem games.

I'm not sure whether on not Nick would have added the requirement
without the added trouble of needing an admin to kill these games (it
sounds like he was a bit disappointed in how bad some programs were at
deciding what was alive and dead), but I'm sure that it helped to push
him towards requiring a no-humans-involved solution.

_______________________________________________
computer-go mailing list
computer-go@xxxxxxxxxxxxxxxxx
http://www.computer-go.org/mailman/listinfo/computer-go/