set the correct loss_improvement for the AverageLearner

[basnijholt]

(original issue on GitLab)

opened by Bas Nijholt (@basnijholt) at 2018-01-11T14:37:14.284Z

Currently test_expected_loss_improvement_is_less_than_total_loss fails because we set an incorrect loss_improvement.

[basnijholt]

originally posted by Bas Nijholt (@basnijholt) at 2018-01-16T12:24:43.955Z on GitLab

I've tested whether the loss improvement implemented in 882cc39ab6efb32e60a79011202bae6dfbf8740b is correct.

%%opts Curve [logy=True]
def g(n):
    return random.gauss(1, 1)

learner = adaptive.AverageLearner(g, atol=None, rtol=0.01)

losses = []
loss_improvements = []
for i in range(10000):
    xs, loss_impr = learner.choose_points(1)
    for x in xs:
        learner.add_point(x, learner.function(x))
    loss_improvements.append(loss_impr[0])
    losses.append(learner.loss())

x = -np.diff(losses)
N = 100  # Running mean over N values
loss = np.convolve(x, np.ones((N,))/N, mode='valid')

hv.Curve(loss) * hv.Curve(loss_improvements)

Definitely looks like it: