[Computer-go] New version of CLOP
sheppardco at aol.com
Mon Nov 7 05:07:27 PST 2011
I have also seen runs where many localization iterations are necessary, but
I did not connect that to overfitting.
I do see Clop runs that "wander" for an excessive number of trials. That has
troubled me, because I would like to use Clop for high dimensional tuning,
but I cannot afford to run 1e7 trials.
Apart from hardware availability, there are two objections to really long
- I am sure to change my program before it finishes the run.
- It might be worse than doing 1e3 runs of 1e4 trials each, using
smaller parameter families.
A lot depends on the correlations between parameters. For example, if
parameters are independent then the potential benefit of tuning all
concurrently is minimized. If the parameters are intricately related, then
there could be great benefit in concurrent tuning, since tuning one
parameter at a time might not see the big picture.
I am curious: can Clop exploit a covariance matrix to "factor" the samples
into minimally interdependent sets? For example, in the case of fully
independent parameters, then Clop could choose x0 by ignoring all
coordinates but x0 in its sample.
The challenge is to create a scalable form of dimensionality reduction.
From: computer-go-bounces at dvandva.org
[mailto:computer-go-bounces at dvandva.org] On Behalf Of Rémi Coulom
Sent: Monday, November 07, 2011 3:34 AM
To: computer-go at dvandva.org
Subject: Re: [Computer-go] New version of CLOP
The prior strength was increased from 1e-3 to 1e-2, and the number of
iterations of the localization process was limited to 7.
I did not have time to measure this very precisely, but tests on Rosenbrock5
show a performance improvement. I made that change because some users of
CLOP sent data to me that show that when CLOP had many parameters (>10), it
had a tendency to overfit: the localization iteration looped until only a
few wins were left. That did not prevent the algorithm from working, but it
was a bit inefficient.
On 7 nov. 2011, at 03:50, Brian Sheppard wrote:
> How does the new version improve regularization in high dimensions? My
> diff of the source code showed some changes, but it was not clear
> which changes improve regularization, specifically in high dimensions.
> Computer-go mailing list
> Computer-go at dvandva.org
Computer-go mailing list
Computer-go at dvandva.org
More information about the Computer-go