[computer-go] Pay-as-you-go cluster?

[David Doshay]

On 10/1/06, Darren Cook <darren at dcook.org> wrote:

> For many of us a cluster is just not practical: even if we could =20
> afford
> 100 computers, where to put them, and how to meet the power
> requirements. But if you have a scalable algorithm then it may be nice
> to have 100 machines for just the couple of hours of a tournament.

=C5On 1, Oct 2006, at 2:30 PM, Mark Boon wrote:

> $10 per hour is going to add up of course.

Never underestimate development time!

> Occasionally my thoughts
> wander to a really distributed Go program, where you can use the power
> of idle computers a la SETI.

This will only work if you have an algorithm that is willing to give =20
up on
nodes that are not reporting. Otherwise you cannot play an interactive
game. In the @home computing model you never know when the owner
will ask for their machine back, and at that time you loose all =20
predictability
about when you will get your cycles again. It is not trivial to =20
distribute a
number of jobs and then decide which of those jobs you can do without.
In the SETI model it does not matter when the answer returns, but in a
realtime game it does.

> But these systems are coming to the home faster than you may think.
> There are already 2 CPUs in both my iMac and MacBook, 4CPU machines
> will be common next year. Sony's next Playstation will have 7 or 8
> CPUs, although a different architecture. So it's worth thinking about
> these things. Will I have a hundred CPUs around the house in a decade?
> Not unthinkable.

Outdated machines just keep piling up around here ... sigh ...

> The difficulty is chopping up the work in big enough chunks to make it
> worthwhile to do the communication with all the computers. And take
> into account the different speeds with which they can communicate, as
> the 2 cores in my Macs will communicate much faster than with each
> other than with another computer.

In our cluster experiments it is clear that for our data chunks (a move
or not too deep lookahead) communication time is very small compared
with computation time, so fast ethernet is quite fine. The ratio is =20
reversed
for MC move generators, so in that case you probably want to merge trees
rather than individual moves.

> I think a lot of exciting possibilities lie in wait for us here, but
> it's going to take time to develop the ideas and algorithms. As usual,
> the software will lag behind the hardware.

Yup.

Cheers,
David