[Computer-go] Practical significance?

Leandro Marcolino sorianom at usc.edu
Sun Nov 25 17:03:33 PST 2012


Hello all!..

I am currently doing a research about Computer Go. I can't tell the details
about it yet, but I will post them here after (if) my paper is accepted...

In my research I compare many systems (An), playing against a fixed strong
adversary (B). So A1 would have a percentage of victory x1 against B, while
A2 would have a percentage of victory x2, etc... Then I compare the
percentage of victories, and for most cases I can show that one system is
better than another with 95% of confidence. However, my adviser is asking
me about not only the STATISTICAL significance of the results, but also the
PRACTICAL significance of them. I mean, if one system is, for example only
1% better than another, with 99% of confidence, the result would have a
statistical significance, but wouldn't really matter in a practical sense.

In my case, the difference between the systems can range from about 4% to
about 23%. Doesn't seem to be enough to argue that one system would be
one-handicap stone better than another. But what would be the minimum
difference for me to argue that one system is significantly better than
another, in a practical sense? (or they are not, in the end?..) Would
calculating ELO-ratings help me in answering this question?

I think it gets even more complex if we think that, let's say, changing the
percentage of victory from 95% to 100% seems to be much more significant
(in a practical sense) than changing from 30% to 35%, even though the
difference between the two systems is still only 5%. In my case, I am
dealing with percentages of victories that range from around 30% to  around
53%.

What do you guys think?..

Thanks for your help!..

Regards,
Leandro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://computer-go.org/pipermail/computer-go/attachments/20121125/3f2139ca/attachment.html>


More information about the Computer-go mailing list