[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [computer-go] 2nd KGS Computer Go Tournament
I think this is perfect for computer/computer play. It's simple, clean
and fair.
- Don
From: "William M. Shubert" <wms@xxxxxxxxxxxxxxxxx>
Date: Mon, 09 May 2005 16:18:27 -0700
Cc:
Reply-To: computer-go <computer-go@xxxxxxxxxxxxxxxxx>
Sender: computer-go-bounces@xxxxxxxxxxxxxxxxx
X-Spam-Score: -4.8
X-Spam-Flag: NO
X-Scanned-By: MIMEDefang 2.42
--===============1600294499==
Content-Type: multipart/alternative; boundary="=-1y+oQumGWbpJJ/vQHGY6"
--=-1y+oQumGWbpJJ/vQHGY6
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
Here's my thoughts on the end-of-game scoring thing.
The fact that the score isn't set in stone after
genmove/pass/genmove_cleanup/pass is because the protocol was intended
for use against humans. The thought was the computer would play until
all dead enemy stones were removed, then pass, and the human had the
choice of removing all dead computer stones or passing. This may be nice
when playing humans, but as was pointed out (first by Don Dailey I
think), in a computer vs. computer tournament it isn't so great.
So here's what I think would work all right. Feedback welcome, I'll give
Nick's opinion the most weight though; since he is doing the organizing
and judging, he's the one who will have to live with the final system
the most!
First, what I think is important, the goals in any change:
1. No program should be forced to implement nonstandard GTP
commands - that means "kgs-genmove_cleanup" has to be optional.
2. Although this system will be sligthly different under tournament
and non-tournament play, programs that always assume KGS GTP
tournament mode should work fine for any GTP game, whether KGS
GTP non-tournament or totally not on KGS at all.
3. A human judge (e.g., Nick) should only have to manage a game
score when programs are buggy, and then they should only have to
assign a win to the non-buggy program. Figuring out which
program is buggy should be easy.
4. It should be easy for programmers to implement.
So, going on these goals, here's what I came up with:
1. To play in a tournament, programs must either implement both
"kgs-genmove_cleanup" and "final_status_list dead", or they must
play until all of their opponent's dead stones are removed from
the board. It's OK if "play until dead stones removed" is an
option, but they have to make sure that this option is turned on
whenever they are going to be in a tournament, or they will do
poorly in the tournament!
2. Programs play as normal.
3. After double pass in a tournament game, programs that support
both "kgs-genmove_cleanup" and "final_status_list dead" will be
sent "final_status_list dead", which will be uploaded to the
server as the list of dead stones. Programs which are missing
either of those from their list of supported commands will tell
the server that no stones are dead.
4. If there is disagreement, "kgs-genmove_cleanup" will be sent to
programs that support it, "genmove" to programs that do not.
Note that there cannot be a disagreement if neither program
supports "kgs-genmove_cleanup" (since after all both will have
reported all stones as alive!), so at least one program will get
the cleanup command.
5. After a second double pass, both programs will report to the
server that no stones are dead, and the server will score from
there.
Look good? I think it should be easy for programmers - if your program
plays at all, you just need a setting "play until all opponent dead
stones are removed" and you're done. If you want to get fance, do
cleanup & final_status_list, but that's optional. Any program that is
buggy (eg, passes at the wrong time with dead stones not removed) will
simply suffer by having the opponent dead stones left on the board, and
that's its own fault. Any program that claims to support cleanup or
final_status_list but returns an error from either of those will be cut
off by the GTP client.
Nick & others, let me know how this looks.
--=-1y+oQumGWbpJJ/vQHGY6
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; CHARSET=UTF-8">
<META NAME="GENERATOR" CONTENT="GtkHTML/3.3.2">
</HEAD>
<BODY>
Here's my thoughts on the end-of-game scoring thing.<BR>
<BR>
The fact that the score isn't set in stone after genmove/pass/genmove_cleanup/pass is because the protocol was intended for use against humans. The thought was the computer would play until all dead enemy stones were removed, then pass, and the human had the choice of removing all dead computer stones or passing. This may be nice when playing humans, but as was pointed out (first by Don Dailey I think), in a computer vs. computer tournament it isn't so great.<BR>
<BR>
So here's what I think would work all right. Feedback welcome, I'll give Nick's opinion the most weight though; since he is doing the organizing and judging, he's the one who will have to live with the final system the most!<BR>
<BR>
First, what I think is important, the goals in any change:
<OL TYPE=1>
<LI TYPE=1 VALUE=1>No program should be forced to implement nonstandard GTP commands - that means "kgs-genmove_cleanup" has to be optional.
<LI TYPE=1 VALUE=2>Although this system will be sligthly different under tournament and non-tournament play, programs that always assume KGS GTP tournament mode should work fine for any GTP game, whether KGS GTP non-tournament or totally not on KGS at all.
<LI TYPE=1 VALUE=3>A human judge (e.g., Nick) should only have to manage a game score when programs are buggy, and then they should only have to assign a win to the non-buggy program. Figuring out which program is buggy should be easy.
<LI TYPE=1 VALUE=4>It should be easy for programmers to implement.
</OL>
So, going on these goals, here's what I came up with:
<OL TYPE=1>
<LI TYPE=1 VALUE=1>To play in a tournament, programs must either implement both "kgs-genmove_cleanup" and "final_status_list dead", <I>or</I> they must play until all of their opponent's dead stones are removed from the board. It's OK if "play until dead stones removed" is an option, but they have to make sure that this option is turned on whenever they are going to be in a tournament, or they will do poorly in the tournament!
<LI TYPE=1 VALUE=2>Programs play as normal.
<LI TYPE=1 VALUE=3>After double pass in a tournament game, programs that support both "kgs-genmove_cleanup" and "final_status_list dead" will be sent "final_status_list dead", which will be uploaded to the server as the list of dead stones. Programs which are missing either of those from their list of supported commands will tell the server that no stones are dead.
<LI TYPE=1 VALUE=4>If there is disagreement, "kgs-genmove_cleanup" will be sent to programs that support it, "genmove" to programs that do not. Note that there cannot be a disagreement if neither program supports "kgs-genmove_cleanup" (since after all both will have reported all stones as alive!), so at least one program will get the cleanup command.
<LI TYPE=1 VALUE=5>After a second double pass, both programs will report to the server that no stones are dead, and the server will score from there.
</OL>
Look good? I think it should be easy for programmers - if your program plays at all, you just need a setting "play until all opponent dead stones are removed" and you're done. If you want to get fance, do cleanup & final_status_list, but that's optional. Any program that is buggy (eg, passes at the wrong time with dead stones not removed) will simply suffer by having the opponent dead stones left on the board, and that's its own fault. Any program that claims to support cleanup or final_status_list but returns an error from either of those will be cut off by the GTP client.<BR>
<BR>
Nick & others, let me know how this looks.
</BODY>
</HTML>
--=-1y+oQumGWbpJJ/vQHGY6--
--===============1600294499==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
computer-go mailing list
computer-go@xxxxxxxxxxxxxxxxx
http://www.computer-go.org/mailman/listinfo/computer-go/
--===============1600294499==--
_______________________________________________
computer-go mailing list
computer-go@xxxxxxxxxxxxxxxxx
http://www.computer-go.org/mailman/listinfo/computer-go/