[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: computer-go: Computer Go hardware



Thanks for the reply David.

So it seems that there is some room for optimism.  When Mark Boon said
only marginal progress has  been made, he clearly  meant not nearly as
much as one would hope, or perhaps very little if any in some areas.

But from what you just said, there has actually been  a fair amount of
progress.  Clearly, with something around 100X faster computers we may
feel that we should be getting more, but how can  anyone know how much
is the "correct" amount?  It is what it is.

And over the  last 10 years  you  and others  have figured  out how to
utilize at  least some of   it.  Ten years from   now, you (or someone
else) will also figure  out how to  utilize  whatever new hardware  we
have.  It might not be better reading, it could be something else.  It
will be whatever is discovered to work best.

There  should be  a much  more  consistant  attempt to  benchmark  our
progress.  It's  like no  one really  knows that todays  programs  are
better.  It's really easy to  complain about how poorly these programs
play and how  far we have  to go, but why  don't we even know how much
progress we have made?

There might  be a lesson to be  learned in this  anecdote from  my own
personal  experiences.  I just  recently   dusted  off my  old   Chess
program, which I considered my best engineering effort.  It's about 10
years old.  I spent an enormous amount of time on it back then, it was
fully debugged and  tested (more than  anything I have today)  and won
tournaments back then.    

Since then I  have   done 3  or 4  rewrites,  some to  remedy  various
weaknesses that  I percieved this  program had and some for commercial
or other reasons.  I  never put as much effort  into these new efforts
as I did this  10 year old program and  never felt these new  programs
reached the same level of "polish."

I was very  much afraid to benchmark it  after all these years, surely
this  "great" program would embarass  me, and prove  to me that I have
wasted  my time trying to  improve   such a  great effort.  On  todays
hardware, this program must really be awesome!

Surprise!  This program is not so good after all.  I had made a lot of
improvments that were immediately obvious!  In 10 years I actually did
learn  something.  Not so   much   me, but  the whole   computer chess
community.   A lot    of  tiny incremental   improvements  made a  big
difference.  There was no  single remarkable difference or improvement
but the point   is that my perceptions  and  expectations have changed
dramatically over the years.   What was great  back then, is  ordinary
now.  I just forgot.

Yes, the same old seemingly unsolvable  problems exist, and no one has
figured out how to  solve them.  Since "familiarity  breeds contempt",
the weakness and problems  of anything you  do stick  out like a  sore
thumb and can make you pessimistic.


Don






   X-Sender: fotland@xxxxxxxxxxxxxxxxx
   X-Mailer: QUALCOMM Windows Eudora Version 5.0.2
   Date: Tue, 10 Apr 2001 23:26:59 -0700
   From: David Fotland <fotland@xxxxxxxxxxxxxxxxx>
   References: <000e01c0c1df$5bd33510$093ba8c0@xxxxxxxxxxxxxxxxx>
   Mime-Version: 1.0
   Sender: owner-computer-go@xxxxxxxxxxxxxxxxx
   Precedence: bulk
   Reply-To: computer-go@xxxxxxxxxxxxxxxxx
   Content-Type: multipart/mixed;
	   boundary="=====================_288238989==_"
   Content-Length: 14394

   --=====================_288238989==_
   Content-Type: text/plain; charset="us-ascii"; format=flowed


   My 2 cents:  I just pulled out the 1990 Goliath and played two games 
   against Many Faces,
   with both colors.  Many Faces won both, by 1.5 and 31.5.  It's too few 
   games to tell, but in
   1990, Goliath was considerably stronger than Many Faces, so I'm satisfied 
   that Many Faces
   has gotten stronger in the last 10 years :)

   Many Faces used about 15 minutes total, and Goliath made most of its moves 
   instantly, with
   a few taking 1 to 2 seconds.  This is on a 450 MHz Pentium-3.

   Then to test the equal-machines idea, I played a game with Many Faces set 
   to level 3, which
   is much, much weaker against people.  Many Faces used 9 seconds for the 
   whole game, and won
   by 12.5 points.

   At 07:36 PM 4/10/2001 -0400, Don Dailey wrote:

   >Hi Mark,
   >
   >You say the strongest programs of today  are only marginally stronger.
   >I'm not in a good position to know if this is true or not, but can you
   >(or someone) try to quantify this a  little better?  I'm interested in
   >a rough   compilation of the  following  information, even if   it's a
   >guess:
   >
   >   1. What do you mean by only mariginally?  In a 100 game match
   >      with no handicap,  would todays very best program have
   >      a good chance of losing to the best 10 year old program on 10 year
   >      old hardware?    What would be the "expected" score?

   Most of todays programs would always lose on 10 year old hardware since 
   they would not make
   the time control.  I don't think this means much though, since the progress 
   in the last few years
   has been in using the faster computer to read better.  There hasn't been 
   much effort to make the
   programs play well on slow hardware :)


   >   2. If I had a competitive program 10 years ago, then I would
   >      not have to maintain it, it would still be competive today
   >      even with that old 10 year old computer.  Is this true or false?

   False.  I think there are  6 to 10 programs today that are stronger than 
   the 10 year
   old Goliath (3 time world champion).


   >   3. Does the hardware make any difference?   Will the newest and
   >      best programs play equally well on ANY old machine, say a 486
   >      class machine (with no extra time allowance of course)?
   >
   >      This is an important point, because if the answer is YES, hardware
   >      makes a difference, and there has been no improvement,  then the
   >      only difference in todays software is that it takes a more powerful
   >      computer to play just as well, a kind of negative progress.

   Absolutely not.  Many Faces can play a game at 100 times the rate of a 
   tournament
   game, but it is much weaker when it does so against people, since it reads much
   more poorly.  Against a program like Goliath that plays good shape and 
   avoids fights,
   reading skill isn't such a big benefit.  Hardware makes a huge difference 
   to fighting strength.



   >In other  messages  from   this group,  I   have been  told   that the
   >programmers have adjusted their algorithms  as hardware has  improved.
   >This directly implies that it's possible  to take advantage, if even a
   >little, of additional   computing power.  And  if  the improvement has
   >only  been  marginal, then it  almost seems  like  all the improvments
   >(what little there has been) is all based on extra computing power.

   Most of the improvement with compute power is in reading skill.  I don't think
   todays programs are much stronger strategically.  But against human opponents,
   or computer opponents that start fights, that reading skill makes a huge 
   difference
   in strength, even if the moves don't look much better in a quiet game.

   David


   >We  could actually do  the  test, I'm sure  someone has  hardware that
   >represents a PC from 1991 and an old program.
   >
   >
   >Don
   >
   >
   >
   >
   >    From: "Mark Boon" <tesuji@xxxxxxxxxxxxxxxxx>
   >    Date: Tue, 10 Apr 2001 18:57:42 +0200
   >    MIME-Version: 1.0
   >    X-Priority: 3 (Normal)
   >    X-MSMail-Priority: Normal
   >    X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
   >    X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
   >    Importance: Normal
   >    Sender: owner-computer-go@xxxxxxxxxxxxxxxxx
   >    Precedence: bulk
   >    Reply-To: computer-go@xxxxxxxxxxxxxxxxx
   >    Content-Type: multipart/alternative;
   >            boundary="----=_NextPart_000_000F_01C0C1F0.1F5C0510"
   >    Content-Length: 4208
   >
   >    This is a multi-part message in MIME format.
   >
   >    ------=_NextPart_000_000F_01C0C1F0.1F5C0510
   >    Content-Type: text/plain;
   >            charset="iso-8859-1"
   >    Content-Transfer-Encoding: 7bit
   >
   >    My 2 cents worth: the strongest programs of today are only marginally
   >    stronger than they were 10 years ago, when computers were 100 times slower
   >    than they are today. So I think it's fair to say the efficiency index 
   > value
   >    would actually be (significantly) lower than 0.01. So your value 0.7 is
   >    quite a wild guess indeed. If I would have to make a wild guess, I'd 
   > rather
   >    put it in the 0.001 to 0.0001 range. And since the programs don't get
   >    stronger with more processing power (not yet, anyway), the index is
   >    subjective to getting halved about every 18 months.
   >
   >        Mark
   >      -----Original Message-----
   >      From: owner-computer-go@xxxxxxxxxxxxxxxxx
   >    [mailto:owner-computer-go@xxxxxxxxxxxxxxxxx]On Behalf Of Compgo123@xxxxxxxxxxxxxxxxx
   >      Sent: Saturday, March 31, 2001 7:23 PM
   >      To: computer-go@xxxxxxxxxxxxxxxxx
   >      Subject: computer-go: Computer Go hardware
   >        [ stuff deleteded ]
   >
   >      We may define a concept, calling it the 'logic
   >      efficiency index'. For a best possible program, the index value is 
   > 1. I'll
   >      make a wild quess here. Today's best program has an index value 
   > about 0.7.
   >    A
   >      related question is, for Go, what's the relation between the index value
   >    and
   >      the amount of programming?
   >      [ stuff deleteded ]
   >
   >
   >    ------=_NextPart_000_000F_01C0C1F0.1F5C0510
   >    Content-Type: text/html;
   >            charset="iso-8859-1"
   >    Content-Transfer-Encoding: quoted-printable
   >
   >    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
   >    <HTML><HEAD>
   >    <META content=3D"text/html; charset=3Dwindows-1252" =
   >    http-equiv=3DContent-Type>
   >    <META content=3D"MSHTML 5.00.2314.1000" name=3DGENERATOR></HEAD>
   >    <BODY>
   >    <DIV><FONT color=3D#0000ff face=3DArial size=3D2><SPAN =
   >    class=3D359254816-10042001>My 2=20
   >    cents worth: the strongest programs of today are only marginally =
   >    stronger than=20
   >    they were 10 years ago, when computers were 100 times slower than they =
   >    are=20
   >    today. So I think it's fair to say&nbsp;the efficiency index value would =
   >
   >    actually be (significantly) lower than 0.01. So your value 0.7 is quite =
   >    a wild=20
   >    guess indeed. If I would have to make a wild guess, I'd rather put it in =
   >    the=20
   >    0.001 to 0.0001 range. And since the programs don't get stronger with =
   >    more=20
   >    processing power (not yet, anyway), the index is subjective to getting =
   >    halved=20
   >    about every 18 months.</SPAN></FONT></DIV>
   >    <DIV><FONT color=3D#0000ff face=3DArial size=3D2><SPAN=20
   >    class=3D359254816-10042001></SPAN></FONT>&nbsp;</DIV>
   >    <DIV><FONT color=3D#0000ff face=3DArial size=3D2><SPAN=20
   >    class=3D359254816-10042001>&nbsp;&nbsp;&nbsp; Mark</SPAN></FONT></DIV>
   >    <BLOCKQUOTE=20
   >    style=3D"BORDER-LEFT: #0000ff 2px solid; MARGIN-LEFT: 5px; PADDING-LEFT: =
   >    5px">
   >      <DIV class=3DOutlookMessageHeader><FONT face=3D"Times New Roman"><FONT =
   >
   >      size=3D2>-----Original Message-----<BR><B>From:</B>=20
   >      owner-computer-go@xxxxxxxxxxxxxxxxx=20
   >      [mailto:owner-computer-go@xxxxxxxxxxxxxxxxx]<B>On Behalf Of</B>=20
   >      Compgo123@xxxxxxxxxxxxxxxxx<BR><B>Sent:</B> Saturday, March 31, 2001 7:23=20
   >      PM<BR><B>To:</B> computer-go@xxxxxxxxxxxxxxxxx<BR><B>Subject:</B> =
   >    computer-go:=20
   >      Computer Go hardware<BR><FONT color=3D#0000ff></FONT><FONT =
   >    size=3D2><FONT=20
   >      face=3DArial><SPAN class=3D359254816-10042001>&nbsp;<FONT =
   >    color=3D#000000=20
   >      face=3D"Times New Roman">&nbsp;[ stuff deleteded=20
   >      ]</FONT>&nbsp;</SPAN><BR></FONT></FONT></FONT></DIV>
   >      <DIV></FONT><FONT face=3DArial><FONT size=3D2>We may define a concept, =
   >    calling it=20
   >      the 'logic <BR>efficiency index'. For a best possible program, the =
   >    index value=20
   >      is 1. I'll <BR>make a wild quess here. Today's best program has an =
   >    index value=20
   >      about 0.7. A <BR>related question is, for Go, what's the relation =
   >    between the=20
   >      index value and <BR>the amount of programming?&nbsp;</FONT>&nbsp;<FONT =
   >
   >      color=3D#0000ff size=3D2><SPAN=20
   >      class=3D359254816-10042001>&nbsp;</SPAN></FONT></FONT></DIV>
   >      <DIV><FONT face=3DArial><FONT color=3D#0000ff size=3D2><SPAN=20
   >      class=3D359254816-10042001>[ stuff deleteded=20
   >      =
   >    ]&nbsp;<BR>&nbsp;</SPAN></FONT></FONT></DIV></BLOCKQUOTE></BODY></HTML>
   >
   >    ------=_NextPart_000_000F_01C0C1F0.1F5C0510--

   David Fotland

   --=====================_288238989==_
   Content-Type: text/plain; charset="us-ascii"
   Content-Disposition: attachment; filename="goliath3.sgf"

   (;
   GM[1]FF[4]VW[]AP[Many Faces of Go:10.0]
   SZ[19]
   HA[0]
   ST[1]
   PB[The Many Faces of Go, level 3, 9 seconds]
   PW[goliath 1990]
   DT[2001-04-10]
   KM[5.5]
   RU[Japanese]
   RE[B+12.5];B[qd];W[po];B[dc];W[ep];B[kq];W[hq];B[cp];W[mq]
   ;B[dn];W[jr];B[qm];W[qk];B[om];W[rn];B[oc];W[ce];B[cg];W[ee]
   ;B[fd];W[dh];B[cd];W[ch];B[be];W[cf];B[bf];W[bg];B[rm];W[ok]
   ;B[ql];W[rk];B[cj];W[ej];B[mm];W[qn];B[km];W[lo];B[jo];W[kr]
   ;B[im];W[no];B[qi];W[mk];B[ml];W[mi];B[gn];W[lr];B[pf];W[lk]
   ;B[jk];W[mf];B[ji];W[mc];B[kc];W[kh];B[hd];W[cl];B[bn];W[dk]
   ;B[rj];W[pi];B[qg];W[ph];B[lb];W[mb];B[gf];W[ki];B[jj];W[cq]
   ;B[bq];W[dq];B[br];W[bl];B[ef];W[dg];B[ne];W[me];B[ke];W[md]
   ;B[bj];W[di];B[eg];W[fe];B[ge];W[bi];B[nb];W[la];B[de];W[df]
   ;B[dd];W[nf];B[ka];W[fh];B[em];W[od];B[pc];W[nd];B[jq];W[iq]
   ;B[es];W[fq];B[gs];W[hr];B[ma];W[dr];B[ds];W[cs];B[cr];W[qh]
   ;B[rh];W[og];B[ig];W[jh];B[eh];W[ei];B[ff];W[gi];B[ih];W[qq]
   ;B[gk];W[pn];B[pm];W[sj];B[ri];W[ip];B[ko];W[ln];B[lm];W[gp]
   ;B[hi];W[gj];B[hj];W[fk];B[gl];W[af];B[bd];W[fl];B[qj];W[pk]
   ;B[ho];W[io];B[in];W[fm];B[fn];W[pj];B[gm];W[nl];B[nm];W[el]
   ;B[kj];W[lj];B[ae];W[ag];B[pe];W[sm];B[oe];W[sl];B[on];W[oo]
   ;B[mn];W[mo];B[lp];W[si];B[rg];W[er];B[bs];W[gr];B[fs];W[cm]
   ;B[cn];W[ld];B[kd];W[dm];B[en];W[sh];B[sg];W[sk];B[jg];W[kg]
   ;B[kf];W[am];B[an];W[ol];B[lf];W[lg];B[jp];W[kn];B[jn];W[go]
   ;B[hs];W[is];B[kl];W[fo];B[do];W[lq];B[bm];W[al];B[dp];W[oq]
   ;B[cs];W[mp];B[kp];W[nc];B[fr];W[eq];B[oa];W[pd];B[rc];W[pg]
   ;B[fg];W[of];B[se];W[gh];B[hg];W[lc];B[gg];W[le];B[hh];W[kk]
   ;B[eo];W[ll];B[pl];W[rl];B[nn];W[hp];B[hn];W[tt];B[tt])

   --=====================_288238989==_
   Content-Type: text/plain; charset="us-ascii"
   Content-Disposition: attachment; filename="goliath2.sgf"

   (;
   GM[1]FF[4]VW[]AP[Many Faces of Go:10.0]
   SZ[19]
   HA[0]
   ST[1]
   PB[Goliath 1990]
   PW[The Many Faces of Go]
   DT[2001-04-10]
   KM[5.5]
   RU[Japanese]
   RE[W+1.5];B[eq];W[pq];B[pc];W[cd];B[ed];W[hd];B[dg];W[kc]
   ;B[cc];W[ci];B[bg];W[qo];B[jq];W[qe];B[qd];W[cl];B[pe];W[nc]
   ;B[nq];W[fp];B[fq];W[co];B[gp];W[cq];B[ep];W[bc];B[dc];W[cb]
   ;B[db];W[ba];B[ab];W[be];B[bb];W[or];B[nr];W[oo];B[qj];W[ql]
   ;B[qg];W[fg];B[di];W[dj];B[ei];W[gh];B[ch];W[bi];B[ej];W[ck]
   ;B[dn];W[cn];B[dm];W[ac];B[ca];W[ob];B[fo];W[ne];B[pb];W[fc]
   ;B[lp];W[ok];B[rk];W[oh];B[rl];W[pi];B[pf];W[of];B[qm];W[qi]
   ;B[ri];W[gj];B[ek];W[rn];B[rm];W[ce];B[ef];W[fe];B[ee];W[gl]
   ;B[ff];W[gf];B[fd];W[gd];B[ge];W[qh];B[rh];W[he];B[ho];W[eb]
   ;B[ke];W[kg];B[ki];W[kk];B[mk];W[ik];B[ii];W[ig];B[lj];W[mm]
   ;B[ll];W[kn];B[nl];W[in];B[ml];W[pl];B[pm];W[ji];B[jj];W[jh]
   ;B[jk];W[kl];B[kj];W[kh];B[ij];W[lm];B[jl];W[km];B[nm];W[nn]
   ;B[om];W[pj];B[lh];W[mi];B[li];W[lg];B[mh];W[ni];B[md];W[nd]
   ;B[je];W[jd];B[mc];W[mb];B[kd];W[lb];B[hb];W[ic];B[dr];W[cr]
   ;B[oa];W[il];B[jm];W[im];B[jn];W[jo];B[lr];W[mg];B[hk];W[gk]
   ;B[io];W[lo];B[mp];W[na];B[pa];W[cm];B[dl];W[hj];B[pg];W[og]
   ;B[ec];W[fb];B[cs];W[bs];B[ds];W[ar];B[hi];W[gi];B[bh];W[qk]
   ;B[rj];W[dq];B[er];W[ib];B[hh];W[hf];B[fe];W[sm];B[ea];W[gb]
   ;B[dk];W[re];B[rd];W[cj];B[oq];W[pp];B[pr];W[qr];B[os];W[qs]
   ;B[nh];W[fa];B[da];W[fl];B[ng];W[nf];B[mo];W[mn];B[ai];W[aj]
   ;B[ah];W[em];B[en];W[kp];B[kq];W[ps];B[or];W[mj];B[lk];W[nk]
   ;B[oc];W[nb];B[od];W[jp];B[ip];W[ko];B[sl];W[sn];B[qn];W[rp]
   ;B[on];W[no];B[do];W[dp];B[me];W[ld];B[lc];W[ol];B[eg];W[eh]
   ;B[hg];W[gg];B[ih];W[if];B[fi];W[fh];B[dh];W[el];B[pn];W[po]
   ;B[np];W[gn];B[fn];W[hn];B[fm];W[gm];B[oe];W[ph];B[fj];W[go]
   ;B[fk];W[hq];B[hp];W[op];B[qf];W[tt])

   --=====================_288238989==_
   Content-Type: text/plain; charset="us-ascii"
   Content-Disposition: attachment; filename="goliath1.sgf"

   (;
   GM[1]FF[4]VW[]AP[Many Faces of Go:10.0]
   SZ[19]
   HA[0]
   ST[1]
   PB[The Many Faces of Go]
   PW[Goliath, 1990]
   DT[2001-04-10]
   KM[5.5]
   RU[Japanese]
   RE[B+31.5];B[qd];W[cd];B[pq];W[co];B[oc];W[po];B[ed];W[cg]
   ;B[cc];W[bc];B[dc];W[bb];B[hc];W[qq];B[ql];W[qp];B[cq];W[ep]
   ;B[eq];W[fq];B[bo];W[bp];B[er];W[fp];B[dp];W[do];B[bq];W[cp]
   ;B[fr];W[pr];B[ph];W[aq];B[hq];W[cl];B[fn];W[go];B[kd];W[fl]
   ;B[gn];W[ho];B[ol];W[fo];B[hn];W[en];B[dh];W[bi];B[ce];W[dd]
   ;B[df];W[dg];B[bd];W[ch];B[eg];W[eh];B[fg];W[fh];B[gg];W[io]
   ;B[jq];W[fm];B[oq];W[or];B[nq];W[mr];B[lq];W[nr];B[mq];W[hi]
   ;B[lr];W[qn];B[rm];W[rn];B[sn];W[so];B[sm];W[ro];B[oe];W[ig]
   ;B[ji];W[hh];B[gh];W[gi];B[im];W[jm];B[jl];W[jn];B[il];W[lm]
   ;B[kg];W[kl];B[kk];W[lk];B[kj];W[lj];B[li];W[mi];B[fi];W[ei]
   ;B[mj];W[mh];B[ml];W[ll];B[mk];W[lh];B[jh];W[ki];B[ij];W[oj]
   ;B[ar];W[ap];B[fj];W[gk];B[ni];W[nh];B[oi];W[lf];B[kf];W[le]
   ;B[ke];W[ld];B[lg];W[mg];B[mf];W[nf];B[me];W[ne];B[md];W[nd]
   ;B[mc];W[nc];B[od];W[nb];B[lc];W[pb];B[qb];W[ob];B[mb];W[pc]
   ;B[qc];W[of];B[pf];W[dq];B[dr];W[br];B[dp];W[gr];B[gq];W[ko]
   ;B[pd];W[no];B[hf];W[kh];B[jg];W[if];B[ie];W[bf];B[pj];W[og]
   ;B[pg];W[gm];B[mo];W[cr];B[dq];W[ds];B[hr];W[pm];B[nn];W[oo]
   ;B[cb];W[qa];B[ra];W[pa];B[rb];W[mn];B[mm];W[ln];B[om];W[qm]
   ;B[rl];W[pl];B[pk];W[ae];B[ad];W[hg];B[ej];W[di];B[ef];W[hk]
   ;B[ik];W[jf];B[lp];W[je];B[id];W[jd];B[jc];W[on];B[be];W[af]
   ;B[cf];W[bg];B[ms];W[ns];B[ls];W[nm];B[nl];W[nn];B[hj];W[gj]
   ;B[na];W[lo];B[mp];W[hl];B[bs];W[as];B[kp];W[ar];B[ip];W[jp]
   ;B[jo];W[oh];B[pp];W[hm];B[in];W[jp];B[ii];W[jo];B[hp];W[ih]
   ;B[np];W[gp];B[op];W[oa];B[ma];W[es];B[gs];W[cs];B[fs];W[dj]
   ;B[tt])

   --=====================_288238989==_--