[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [computer-go] 2nd KGS Computer Go Tournament

Here's my thoughts on the end-of-game scoring thing.

The fact that the score isn't set in stone after genmove/pass/genmove_cleanup/pass is because the protocol was intended for use against humans. The thought was the computer would play until all dead enemy stones were removed, then pass, and the human had the choice of removing all dead computer stones or passing. This may be nice when playing humans, but as was pointed out (first by Don Dailey I think), in a computer vs. computer tournament it isn't so great.

So here's what I think would work all right. Feedback welcome, I'll give Nick's opinion the most weight though; since he is doing the organizing and judging, he's the one who will have to live with the final system the most!

First, what I think is important, the goals in any change:
  1. No program should be forced to implement nonstandard GTP commands - that means "kgs-genmove_cleanup" has to be optional.
  2. Although this system will be sligthly different under tournament and non-tournament play, programs that always assume KGS GTP tournament mode should work fine for any GTP game, whether KGS GTP non-tournament or totally not on KGS at all.
  3. A human judge (e.g., Nick) should only have to manage a game score when programs are buggy, and then they should only have to assign a win to the non-buggy program. Figuring out which program is buggy should be easy.
  4. It should be easy for programmers to implement.
So, going on these goals, here's what I came up with:
  1. To play in a tournament, programs must either implement both "kgs-genmove_cleanup" and "final_status_list dead", or they must play until all of their opponent's dead stones are removed from the board. It's OK if "play until dead stones removed" is an option, but they have to make sure that this option is turned on whenever they are going to be in a tournament, or they will do poorly in the tournament!
  2. Programs play as normal.
  3. After double pass in a tournament game, programs that support both "kgs-genmove_cleanup" and "final_status_list dead" will be sent "final_status_list dead", which will be uploaded to the server as the list of dead stones. Programs which are missing either of those from their list of supported commands will tell the server that no stones are dead.
  4. If there is disagreement, "kgs-genmove_cleanup" will be sent to programs that support it, "genmove" to programs that do not. Note that there cannot be a disagreement if neither program supports "kgs-genmove_cleanup" (since after all both will have reported all stones as alive!), so at least one program will get the cleanup command.
  5. After a second double pass, both programs will report to the server that no stones are dead, and the server will score from there.
Look good? I think it should be easy for programmers - if your program plays at all, you just need a setting "play until all opponent dead stones are removed" and you're done. If you want to get fance, do cleanup & final_status_list, but that's optional. Any program that is buggy (eg, passes at the wrong time with dead stones not removed) will simply suffer by having the opponent dead stones left on the board, and that's its own fault. Any program that claims to support cleanup or final_status_list but returns an error from either of those will be cut off by the GTP client.

Nick & others, let me know how this looks. 
_______________________________________________
computer-go mailing list
computer-go@xxxxxxxxxxxxxxxxx
http://www.computer-go.org/mailman/listinfo/computer-go/