[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: computer-go: Computer Go hardware
Thanks for the reply David.
So it seems that there is some room for optimism. When Mark Boon said
only marginal progress has been made, he clearly meant not nearly as
much as one would hope, or perhaps very little if any in some areas.
But from what you just said, there has actually been a fair amount of
progress. Clearly, with something around 100X faster computers we may
feel that we should be getting more, but how can anyone know how much
is the "correct" amount? It is what it is.
And over the last 10 years you and others have figured out how to
utilize at least some of it. Ten years from now, you (or someone
else) will also figure out how to utilize whatever new hardware we
have. It might not be better reading, it could be something else. It
will be whatever is discovered to work best.
There should be a much more consistant attempt to benchmark our
progress. It's like no one really knows that todays programs are
better. It's really easy to complain about how poorly these programs
play and how far we have to go, but why don't we even know how much
progress we have made?
There might be a lesson to be learned in this anecdote from my own
personal experiences. I just recently dusted off my old Chess
program, which I considered my best engineering effort. It's about 10
years old. I spent an enormous amount of time on it back then, it was
fully debugged and tested (more than anything I have today) and won
tournaments back then.
Since then I have done 3 or 4 rewrites, some to remedy various
weaknesses that I percieved this program had and some for commercial
or other reasons. I never put as much effort into these new efforts
as I did this 10 year old program and never felt these new programs
reached the same level of "polish."
I was very much afraid to benchmark it after all these years, surely
this "great" program would embarass me, and prove to me that I have
wasted my time trying to improve such a great effort. On todays
hardware, this program must really be awesome!
Surprise! This program is not so good after all. I had made a lot of
improvments that were immediately obvious! In 10 years I actually did
learn something. Not so much me, but the whole computer chess
community. A lot of tiny incremental improvements made a big
difference. There was no single remarkable difference or improvement
but the point is that my perceptions and expectations have changed
dramatically over the years. What was great back then, is ordinary
now. I just forgot.
Yes, the same old seemingly unsolvable problems exist, and no one has
figured out how to solve them. Since "familiarity breeds contempt",
the weakness and problems of anything you do stick out like a sore
thumb and can make you pessimistic.
Don
X-Sender: fotland@xxxxxxxxxxxxxxxxx
X-Mailer: QUALCOMM Windows Eudora Version 5.0.2
Date: Tue, 10 Apr 2001 23:26:59 -0700
From: David Fotland <fotland@xxxxxxxxxxxxxxxxx>
References: <000e01c0c1df$5bd33510$093ba8c0@xxxxxxxxxxxxxxxxx>
Mime-Version: 1.0
Sender: owner-computer-go@xxxxxxxxxxxxxxxxx
Precedence: bulk
Reply-To: computer-go@xxxxxxxxxxxxxxxxx
Content-Type: multipart/mixed;
boundary="=====================_288238989==_"
Content-Length: 14394
--=====================_288238989==_
Content-Type: text/plain; charset="us-ascii"; format=flowed
My 2 cents: I just pulled out the 1990 Goliath and played two games
against Many Faces,
with both colors. Many Faces won both, by 1.5 and 31.5. It's too few
games to tell, but in
1990, Goliath was considerably stronger than Many Faces, so I'm satisfied
that Many Faces
has gotten stronger in the last 10 years :)
Many Faces used about 15 minutes total, and Goliath made most of its moves
instantly, with
a few taking 1 to 2 seconds. This is on a 450 MHz Pentium-3.
Then to test the equal-machines idea, I played a game with Many Faces set
to level 3, which
is much, much weaker against people. Many Faces used 9 seconds for the
whole game, and won
by 12.5 points.
At 07:36 PM 4/10/2001 -0400, Don Dailey wrote:
>Hi Mark,
>
>You say the strongest programs of today are only marginally stronger.
>I'm not in a good position to know if this is true or not, but can you
>(or someone) try to quantify this a little better? I'm interested in
>a rough compilation of the following information, even if it's a
>guess:
>
> 1. What do you mean by only mariginally? In a 100 game match
> with no handicap, would todays very best program have
> a good chance of losing to the best 10 year old program on 10 year
> old hardware? What would be the "expected" score?
Most of todays programs would always lose on 10 year old hardware since
they would not make
the time control. I don't think this means much though, since the progress
in the last few years
has been in using the faster computer to read better. There hasn't been
much effort to make the
programs play well on slow hardware :)
> 2. If I had a competitive program 10 years ago, then I would
> not have to maintain it, it would still be competive today
> even with that old 10 year old computer. Is this true or false?
False. I think there are 6 to 10 programs today that are stronger than
the 10 year
old Goliath (3 time world champion).
> 3. Does the hardware make any difference? Will the newest and
> best programs play equally well on ANY old machine, say a 486
> class machine (with no extra time allowance of course)?
>
> This is an important point, because if the answer is YES, hardware
> makes a difference, and there has been no improvement, then the
> only difference in todays software is that it takes a more powerful
> computer to play just as well, a kind of negative progress.
Absolutely not. Many Faces can play a game at 100 times the rate of a
tournament
game, but it is much weaker when it does so against people, since it reads much
more poorly. Against a program like Goliath that plays good shape and
avoids fights,
reading skill isn't such a big benefit. Hardware makes a huge difference
to fighting strength.
>In other messages from this group, I have been told that the
>programmers have adjusted their algorithms as hardware has improved.
>This directly implies that it's possible to take advantage, if even a
>little, of additional computing power. And if the improvement has
>only been marginal, then it almost seems like all the improvments
>(what little there has been) is all based on extra computing power.
Most of the improvement with compute power is in reading skill. I don't think
todays programs are much stronger strategically. But against human opponents,
or computer opponents that start fights, that reading skill makes a huge
difference
in strength, even if the moves don't look much better in a quiet game.
David
>We could actually do the test, I'm sure someone has hardware that
>represents a PC from 1991 and an old program.
>
>
>Don
>
>
>
>
> From: "Mark Boon" <tesuji@xxxxxxxxxxxxxxxxx>
> Date: Tue, 10 Apr 2001 18:57:42 +0200
> MIME-Version: 1.0
> X-Priority: 3 (Normal)
> X-MSMail-Priority: Normal
> X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
> X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
> Importance: Normal
> Sender: owner-computer-go@xxxxxxxxxxxxxxxxx
> Precedence: bulk
> Reply-To: computer-go@xxxxxxxxxxxxxxxxx
> Content-Type: multipart/alternative;
> boundary="----=_NextPart_000_000F_01C0C1F0.1F5C0510"
> Content-Length: 4208
>
> This is a multi-part message in MIME format.
>
> ------=_NextPart_000_000F_01C0C1F0.1F5C0510
> Content-Type: text/plain;
> charset="iso-8859-1"
> Content-Transfer-Encoding: 7bit
>
> My 2 cents worth: the strongest programs of today are only marginally
> stronger than they were 10 years ago, when computers were 100 times slower
> than they are today. So I think it's fair to say the efficiency index
> value
> would actually be (significantly) lower than 0.01. So your value 0.7 is
> quite a wild guess indeed. If I would have to make a wild guess, I'd
> rather
> put it in the 0.001 to 0.0001 range. And since the programs don't get
> stronger with more processing power (not yet, anyway), the index is
> subjective to getting halved about every 18 months.
>
> Mark
> -----Original Message-----
> From: owner-computer-go@xxxxxxxxxxxxxxxxx
> [mailto:owner-computer-go@xxxxxxxxxxxxxxxxx]On Behalf Of Compgo123@xxxxxxxxxxxxxxxxx
> Sent: Saturday, March 31, 2001 7:23 PM
> To: computer-go@xxxxxxxxxxxxxxxxx
> Subject: computer-go: Computer Go hardware
> [ stuff deleteded ]
>
> We may define a concept, calling it the 'logic
> efficiency index'. For a best possible program, the index value is
> 1. I'll
> make a wild quess here. Today's best program has an index value
> about 0.7.
> A
> related question is, for Go, what's the relation between the index value
> and
> the amount of programming?
> [ stuff deleteded ]
>
>
> ------=_NextPart_000_000F_01C0C1F0.1F5C0510
> Content-Type: text/html;
> charset="iso-8859-1"
> Content-Transfer-Encoding: quoted-printable
>
> <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
> <HTML><HEAD>
> <META content=3D"text/html; charset=3Dwindows-1252" =
> http-equiv=3DContent-Type>
> <META content=3D"MSHTML 5.00.2314.1000" name=3DGENERATOR></HEAD>
> <BODY>
> <DIV><FONT color=3D#0000ff face=3DArial size=3D2><SPAN =
> class=3D359254816-10042001>My 2=20
> cents worth: the strongest programs of today are only marginally =
> stronger than=20
> they were 10 years ago, when computers were 100 times slower than they =
> are=20
> today. So I think it's fair to say the efficiency index value would =
>
> actually be (significantly) lower than 0.01. So your value 0.7 is quite =
> a wild=20
> guess indeed. If I would have to make a wild guess, I'd rather put it in =
> the=20
> 0.001 to 0.0001 range. And since the programs don't get stronger with =
> more=20
> processing power (not yet, anyway), the index is subjective to getting =
> halved=20
> about every 18 months.</SPAN></FONT></DIV>
> <DIV><FONT color=3D#0000ff face=3DArial size=3D2><SPAN=20
> class=3D359254816-10042001></SPAN></FONT> </DIV>
> <DIV><FONT color=3D#0000ff face=3DArial size=3D2><SPAN=20
> class=3D359254816-10042001> Mark</SPAN></FONT></DIV>
> <BLOCKQUOTE=20
> style=3D"BORDER-LEFT: #0000ff 2px solid; MARGIN-LEFT: 5px; PADDING-LEFT: =
> 5px">
> <DIV class=3DOutlookMessageHeader><FONT face=3D"Times New Roman"><FONT =
>
> size=3D2>-----Original Message-----<BR><B>From:</B>=20
> owner-computer-go@xxxxxxxxxxxxxxxxx=20
> [mailto:owner-computer-go@xxxxxxxxxxxxxxxxx]<B>On Behalf Of</B>=20
> Compgo123@xxxxxxxxxxxxxxxxx<BR><B>Sent:</B> Saturday, March 31, 2001 7:23=20
> PM<BR><B>To:</B> computer-go@xxxxxxxxxxxxxxxxx<BR><B>Subject:</B> =
> computer-go:=20
> Computer Go hardware<BR><FONT color=3D#0000ff></FONT><FONT =
> size=3D2><FONT=20
> face=3DArial><SPAN class=3D359254816-10042001> <FONT =
> color=3D#000000=20
> face=3D"Times New Roman"> [ stuff deleteded=20
> ]</FONT> </SPAN><BR></FONT></FONT></FONT></DIV>
> <DIV></FONT><FONT face=3DArial><FONT size=3D2>We may define a concept, =
> calling it=20
> the 'logic <BR>efficiency index'. For a best possible program, the =
> index value=20
> is 1. I'll <BR>make a wild quess here. Today's best program has an =
> index value=20
> about 0.7. A <BR>related question is, for Go, what's the relation =
> between the=20
> index value and <BR>the amount of programming? </FONT> <FONT =
>
> color=3D#0000ff size=3D2><SPAN=20
> class=3D359254816-10042001> </SPAN></FONT></FONT></DIV>
> <DIV><FONT face=3DArial><FONT color=3D#0000ff size=3D2><SPAN=20
> class=3D359254816-10042001>[ stuff deleteded=20
> =
> ] <BR> </SPAN></FONT></FONT></DIV></BLOCKQUOTE></BODY></HTML>
>
> ------=_NextPart_000_000F_01C0C1F0.1F5C0510--
David Fotland
--=====================_288238989==_
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: attachment; filename="goliath3.sgf"
(;
GM[1]FF[4]VW[]AP[Many Faces of Go:10.0]
SZ[19]
HA[0]
ST[1]
PB[The Many Faces of Go, level 3, 9 seconds]
PW[goliath 1990]
DT[2001-04-10]
KM[5.5]
RU[Japanese]
RE[B+12.5];B[qd];W[po];B[dc];W[ep];B[kq];W[hq];B[cp];W[mq]
;B[dn];W[jr];B[qm];W[qk];B[om];W[rn];B[oc];W[ce];B[cg];W[ee]
;B[fd];W[dh];B[cd];W[ch];B[be];W[cf];B[bf];W[bg];B[rm];W[ok]
;B[ql];W[rk];B[cj];W[ej];B[mm];W[qn];B[km];W[lo];B[jo];W[kr]
;B[im];W[no];B[qi];W[mk];B[ml];W[mi];B[gn];W[lr];B[pf];W[lk]
;B[jk];W[mf];B[ji];W[mc];B[kc];W[kh];B[hd];W[cl];B[bn];W[dk]
;B[rj];W[pi];B[qg];W[ph];B[lb];W[mb];B[gf];W[ki];B[jj];W[cq]
;B[bq];W[dq];B[br];W[bl];B[ef];W[dg];B[ne];W[me];B[ke];W[md]
;B[bj];W[di];B[eg];W[fe];B[ge];W[bi];B[nb];W[la];B[de];W[df]
;B[dd];W[nf];B[ka];W[fh];B[em];W[od];B[pc];W[nd];B[jq];W[iq]
;B[es];W[fq];B[gs];W[hr];B[ma];W[dr];B[ds];W[cs];B[cr];W[qh]
;B[rh];W[og];B[ig];W[jh];B[eh];W[ei];B[ff];W[gi];B[ih];W[qq]
;B[gk];W[pn];B[pm];W[sj];B[ri];W[ip];B[ko];W[ln];B[lm];W[gp]
;B[hi];W[gj];B[hj];W[fk];B[gl];W[af];B[bd];W[fl];B[qj];W[pk]
;B[ho];W[io];B[in];W[fm];B[fn];W[pj];B[gm];W[nl];B[nm];W[el]
;B[kj];W[lj];B[ae];W[ag];B[pe];W[sm];B[oe];W[sl];B[on];W[oo]
;B[mn];W[mo];B[lp];W[si];B[rg];W[er];B[bs];W[gr];B[fs];W[cm]
;B[cn];W[ld];B[kd];W[dm];B[en];W[sh];B[sg];W[sk];B[jg];W[kg]
;B[kf];W[am];B[an];W[ol];B[lf];W[lg];B[jp];W[kn];B[jn];W[go]
;B[hs];W[is];B[kl];W[fo];B[do];W[lq];B[bm];W[al];B[dp];W[oq]
;B[cs];W[mp];B[kp];W[nc];B[fr];W[eq];B[oa];W[pd];B[rc];W[pg]
;B[fg];W[of];B[se];W[gh];B[hg];W[lc];B[gg];W[le];B[hh];W[kk]
;B[eo];W[ll];B[pl];W[rl];B[nn];W[hp];B[hn];W[tt];B[tt])
--=====================_288238989==_
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: attachment; filename="goliath2.sgf"
(;
GM[1]FF[4]VW[]AP[Many Faces of Go:10.0]
SZ[19]
HA[0]
ST[1]
PB[Goliath 1990]
PW[The Many Faces of Go]
DT[2001-04-10]
KM[5.5]
RU[Japanese]
RE[W+1.5];B[eq];W[pq];B[pc];W[cd];B[ed];W[hd];B[dg];W[kc]
;B[cc];W[ci];B[bg];W[qo];B[jq];W[qe];B[qd];W[cl];B[pe];W[nc]
;B[nq];W[fp];B[fq];W[co];B[gp];W[cq];B[ep];W[bc];B[dc];W[cb]
;B[db];W[ba];B[ab];W[be];B[bb];W[or];B[nr];W[oo];B[qj];W[ql]
;B[qg];W[fg];B[di];W[dj];B[ei];W[gh];B[ch];W[bi];B[ej];W[ck]
;B[dn];W[cn];B[dm];W[ac];B[ca];W[ob];B[fo];W[ne];B[pb];W[fc]
;B[lp];W[ok];B[rk];W[oh];B[rl];W[pi];B[pf];W[of];B[qm];W[qi]
;B[ri];W[gj];B[ek];W[rn];B[rm];W[ce];B[ef];W[fe];B[ee];W[gl]
;B[ff];W[gf];B[fd];W[gd];B[ge];W[qh];B[rh];W[he];B[ho];W[eb]
;B[ke];W[kg];B[ki];W[kk];B[mk];W[ik];B[ii];W[ig];B[lj];W[mm]
;B[ll];W[kn];B[nl];W[in];B[ml];W[pl];B[pm];W[ji];B[jj];W[jh]
;B[jk];W[kl];B[kj];W[kh];B[ij];W[lm];B[jl];W[km];B[nm];W[nn]
;B[om];W[pj];B[lh];W[mi];B[li];W[lg];B[mh];W[ni];B[md];W[nd]
;B[je];W[jd];B[mc];W[mb];B[kd];W[lb];B[hb];W[ic];B[dr];W[cr]
;B[oa];W[il];B[jm];W[im];B[jn];W[jo];B[lr];W[mg];B[hk];W[gk]
;B[io];W[lo];B[mp];W[na];B[pa];W[cm];B[dl];W[hj];B[pg];W[og]
;B[ec];W[fb];B[cs];W[bs];B[ds];W[ar];B[hi];W[gi];B[bh];W[qk]
;B[rj];W[dq];B[er];W[ib];B[hh];W[hf];B[fe];W[sm];B[ea];W[gb]
;B[dk];W[re];B[rd];W[cj];B[oq];W[pp];B[pr];W[qr];B[os];W[qs]
;B[nh];W[fa];B[da];W[fl];B[ng];W[nf];B[mo];W[mn];B[ai];W[aj]
;B[ah];W[em];B[en];W[kp];B[kq];W[ps];B[or];W[mj];B[lk];W[nk]
;B[oc];W[nb];B[od];W[jp];B[ip];W[ko];B[sl];W[sn];B[qn];W[rp]
;B[on];W[no];B[do];W[dp];B[me];W[ld];B[lc];W[ol];B[eg];W[eh]
;B[hg];W[gg];B[ih];W[if];B[fi];W[fh];B[dh];W[el];B[pn];W[po]
;B[np];W[gn];B[fn];W[hn];B[fm];W[gm];B[oe];W[ph];B[fj];W[go]
;B[fk];W[hq];B[hp];W[op];B[qf];W[tt])
--=====================_288238989==_
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: attachment; filename="goliath1.sgf"
(;
GM[1]FF[4]VW[]AP[Many Faces of Go:10.0]
SZ[19]
HA[0]
ST[1]
PB[The Many Faces of Go]
PW[Goliath, 1990]
DT[2001-04-10]
KM[5.5]
RU[Japanese]
RE[B+31.5];B[qd];W[cd];B[pq];W[co];B[oc];W[po];B[ed];W[cg]
;B[cc];W[bc];B[dc];W[bb];B[hc];W[qq];B[ql];W[qp];B[cq];W[ep]
;B[eq];W[fq];B[bo];W[bp];B[er];W[fp];B[dp];W[do];B[bq];W[cp]
;B[fr];W[pr];B[ph];W[aq];B[hq];W[cl];B[fn];W[go];B[kd];W[fl]
;B[gn];W[ho];B[ol];W[fo];B[hn];W[en];B[dh];W[bi];B[ce];W[dd]
;B[df];W[dg];B[bd];W[ch];B[eg];W[eh];B[fg];W[fh];B[gg];W[io]
;B[jq];W[fm];B[oq];W[or];B[nq];W[mr];B[lq];W[nr];B[mq];W[hi]
;B[lr];W[qn];B[rm];W[rn];B[sn];W[so];B[sm];W[ro];B[oe];W[ig]
;B[ji];W[hh];B[gh];W[gi];B[im];W[jm];B[jl];W[jn];B[il];W[lm]
;B[kg];W[kl];B[kk];W[lk];B[kj];W[lj];B[li];W[mi];B[fi];W[ei]
;B[mj];W[mh];B[ml];W[ll];B[mk];W[lh];B[jh];W[ki];B[ij];W[oj]
;B[ar];W[ap];B[fj];W[gk];B[ni];W[nh];B[oi];W[lf];B[kf];W[le]
;B[ke];W[ld];B[lg];W[mg];B[mf];W[nf];B[me];W[ne];B[md];W[nd]
;B[mc];W[nc];B[od];W[nb];B[lc];W[pb];B[qb];W[ob];B[mb];W[pc]
;B[qc];W[of];B[pf];W[dq];B[dr];W[br];B[dp];W[gr];B[gq];W[ko]
;B[pd];W[no];B[hf];W[kh];B[jg];W[if];B[ie];W[bf];B[pj];W[og]
;B[pg];W[gm];B[mo];W[cr];B[dq];W[ds];B[hr];W[pm];B[nn];W[oo]
;B[cb];W[qa];B[ra];W[pa];B[rb];W[mn];B[mm];W[ln];B[om];W[qm]
;B[rl];W[pl];B[pk];W[ae];B[ad];W[hg];B[ej];W[di];B[ef];W[hk]
;B[ik];W[jf];B[lp];W[je];B[id];W[jd];B[jc];W[on];B[be];W[af]
;B[cf];W[bg];B[ms];W[ns];B[ls];W[nm];B[nl];W[nn];B[hj];W[gj]
;B[na];W[lo];B[mp];W[hl];B[bs];W[as];B[kp];W[ar];B[ip];W[jp]
;B[jo];W[oh];B[pp];W[hm];B[in];W[jp];B[ii];W[jo];B[hp];W[ih]
;B[np];W[gp];B[op];W[oa];B[ma];W[es];B[gs];W[cs];B[fs];W[dj]
;B[tt])
--=====================_288238989==_--