# [Computer-go] Monte-carlo simulations vs. MCMC

uurtamo . uurtamo at gmail.com
Fri Nov 1 12:53:47 PDT 2013

```A Markov chain is a very specific model type. If you don't model your
problem as a Markov chain, mcmc is irrelevant.

Don't be fooled by the fact that it's "Monte Carlo" - anything from
gradient descent to uniform sampling can use Monte Carlo methods. Or not!

I recommend reading up on machine learning. Stanford has a very nice series
of online lectures that comprise a healthy course of study.

s.
On Nov 1, 2013 12:45 PM, "Darren Cook" <darren at dcook.org> wrote:

> > MCMC has little to do with what we do in computer Go. In MCTS we have
> > a Markov Chain and we take Monte-Carlo samples from it, but the
> > purpose is really not the same at all as what MCMC algorithms do. I
> > recommend the wikipedia articles. It is difficult to really get an
> > idea of MCMC by reading a general description. It is probably best
> > that you get a feeling of what it is by studying the details of a
> > real MCMC algorithm. The most basic MCMC algorithm is the
> > Metropolis-Hastings algorithm:
> > https://en.wikipedia.org/wiki/Metropolis%E2%80%93Hastings_algorithm
>
> Thanks Remi.
> Quoting from that article:
> "...[a] method for obtaining a sequence of random samples from a
> probability distribution for which direct sampling is difficult. This
> sequence can be used to approximate the distribution (i.e., to generate
> a histogram)"
>
> This sounds exactly like using N monte-carlo simulations at a node in an
> MCTS tree, generating a histograms of possible scores. The highest point
> on the histogram is used as the best-guess estimate of the score. When
> you have two peaks it implies some unstable situation, like a semeai. Etc.
>
> You mentioned the "purpose" is not the same. Can you elaborate?
>
> (If "Markov-Chain" is a nice clean synonym for "rules of the game",
> whether the game is go or weather systems, I feel I am on home ground!)
>
> Darren
> _______________________________________________
> Computer-go mailing list
> Computer-go at dvandva.org
> http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://computer-go.org/pipermail/computer-go/attachments/20131101/5b5c4390/attachment.html>
```