Re: Ratings on go servers

[David Fotland <fotland@xxxxxxxxxxxxxxx>]

At 12:54 PM 8/19/99, Darren Cook wrote:
>>I'm doing all my tuning now using games lost on the servers rather 
>>than tuning against other programs.
>
>That's interesting. How automatic is your tuning process?
>
>E.g. at one extreme you look at the games, use your own judgement to pick
>out a handful of bad moves (which I'd guess would often cluster because
>they'd be based on a bad life/death decision), then use your knowledge of
>the program to go into the code and change/add something. Then retest on
>that position to check your improvement worked.

This is what I'm doing.  Often I make a change with the pattern editor rather 
than with code.  The other tuning I do is looking at problems it gets a
wrong answer
for.  I have 1657 go problems that it can try to solve in a few hours.  It
only
gets 1386 correct, and it probably takes me 15-30 minutes on average to fix
the
issue that is making one problem fail, so I can work on problems about
forever.
Of course sometimes a fix helps more than one problem, but sometimes it
makes a bunch go bad as well.

I have no faith in automatic tuning.

IGS ratings seem stable enough for real testing.  I broke something a while
back
and the rating quickly fell from 14k* to 17k*.

David

>
>At the other extreme, and from the other direction, you create a set of
>programs that vary in one parameter. Then play N games with each version of
>the program, and see which has the highest winning ratio. IIRC, Mick is
>doing something like this with Go4++. I'm not sure it is workable with the
>servers though.
>
>My own program (which has sadly been in deep-freeze for the past few
>months) trains by learning the patterns for the opponents moves in games it
>loses. The theory being that then it will know the correct counter-attack
>when it is doing search, and will hopefully then realize it has to avoid
>that branch. One problem with this is that it also learns opponent
>mistakes, which can pollute the database.
>
>Darren
>
>
>
>
>