[computer-go] BayesianElo (was: Re: ZG1-trunMCx-50k)

[Aloril]

I tried BayesianElo program by Rémi Coulom
at http://remi.coulom.free.fr/Bayesian-Elo/

I used this script:
readpgn result.pgn
elo
  advantage 0.0
  mm
  exactdist
  offset 1117
  ratings
  x
x

results.pgn looks like this:
[White "AnchorMan"]
[Black "ReadyToGo"]
[Result "1-0"]

1-0

[White "ReadyToGo"]
[Black "AnchorMan"]
[Result "0-1"]

0-1

etc...


Result of above is at
http://londerings.sourceforge.net/go/cgos/bayesian_elo_cgos.txt
I have added CGOS given ratings and difference between bayesian elo and
CGOS columns.


On Sun, 2006-09-24 at 11:43 +0200, Łukasz Lew wrote:
> Some time ago I put 4 identical bots on CGOS. Now they got stable rank:
> ZG1-trunMC1-50k	1519*
> ZG1-trunMC2-50k	1527*
> ZG1-trunMC3-50k	1529*
> ZG1-trunMC4-50k	1574*	
> 
> They are identiacl to bot that played earlier:
> ZG1-trunMC-100k	1530*
> 
> I would be very happy to have this precision of strength measurement,
> but ZG1-trunMC4-50k has oddly high rating.
> 
> I double checked the binary and parameters of all the bots and they
> are truly the same.
> 
> So what may be the reason of this?
> 
> 
> Lukasz Lew

Too few games. Even using bayesian elo there is difference:
Rank Name               Elo    +    - games score oppo.  CGOS  diff
  95 ZG1-trunMC4-50k   1565   37   37   424   60%  1384  1576*  -11
 106 ZG1-trunMC3-50k   1530   37   37   439   60%  1326  1535*   -5
 108 ZG1-trunMC-100k   1523   29   29   716   56%  1372  1530*   -7
 109 ZG1-trunMC1-50k   1514   36   36   464   58%  1343  1511*    3
 110 ZG1-trunMC2-50k   1502   37   37   434   56%  1379  1525*  -23

Difference is 63 which is less than uncertainty 37*2=74.

For players that are equal and have played a lot rating is close using bayesian elo:

 111 AnchorMan         1500    5    5 41151   60%  1284  1500*    0
 112 ControlBoy        1495    5    6 36712   58%  1296  1486*    9

> 
> 
> On 9/21/06, Łukasz Lew <lukasz.lew at gmail.com> wrote:
> > All CGOS bots
> > ZG1-trunMCx-50k ( x = 1,2,3,4 )
> > are the same program as
> > ZG1-trunMC-100k
> > (also play 100k games not 50k - my mistake)
> >
> > It is an experiment to check drift and accuracy of CGOS rating.
> >
> > Best regards,
> > Lukasz
> >
> > PS
> > Is there a possibility to extend the limit on bot name's length?
> >
> _______________________________________________
> computer-go mailing list
> computer-go at computer-go.org
> http://www.computer-go.org/mailman/listinfo/computer-go/
-- 
Aloril <aloril at iki.fi>