Runners should tell learner about remaining points at end of run

[jgukelberger]

Currently, BlockingRunner will wait for all futures that are running when the goal is reached, but then ignore their results. Looking at the code, AsyncRunner has the same behaviour.

This seems like a waste. I think it should just self._process_futures(remaining) after the wait.

[akhmerov]

Sounds like a great idea!

[basnijholt]

Thanks for reporting!

I checked the remaining variable with the following patch:

diff --git a/adaptive/runner.py b/adaptive/runner.py
index 26a39ca..961f2e4 100644
--- a/adaptive/runner.py
+++ b/adaptive/runner.py
@@ -409,9 +409,9 @@ class BlockingRunner(BaseRunner):
                 done, _ = concurrent.wait(futures, return_when=first_completed)
                 self._process_futures(done)
         finally:
-            remaining = self._remove_unfinished()
-            if remaining:
-                concurrent.wait(remaining)
+            self.remaining = self._remove_unfinished()
+            if self.remaining:
+                concurrent.wait(self.remaining)
             self._cleanup()

     def elapsed_time(self):
@@ -643,9 +643,9 @@ class AsyncRunner(BaseRunner):
                 )
                 self._process_futures(done)
         finally:
-            remaining = self._remove_unfinished()
-            if remaining:
-                await asyncio.wait(remaining)
+            self.remaining = self._remove_unfinished()
+            if self.remaining:
+                await asyncio.wait(self.remaining)
             self._cleanup()

     def elapsed_time(self):

The BlockingRunner:

runner = adaptive.BlockingRunner(learner, goal=lambda l: l.loss() < 0.01)
print(runner.remaining)

prints

[<Future at 0x7f91461b2bb0 state=finished returned float>, <Future at 0x7f91465e7550 state=finished returned float>, <Future at 0x7f91461cab50 state=finished returned float>, <Future at 0x7f9146216c70 state=finished returned float>, <Future at 0x7f91462163a0 state=finished returned float>, <Future at 0x7f91462164c0 state=finished returned float>, <Future at 0x7f91461b2040 state=finished returned float>]

and they contain the proper fut.result().

The AsyncRunner:

runner = adaptive.Runner(learner, goal=lambda l: l.loss() < 0.01)
await runner.task
print(runner.remaining)

prints

[<Future cancelled>, <Future cancelled>, <Future cancelled>, <Future cancelled>, <Future cancelled>, <Future cancelled>]

and fut.result() returns:

---------------------------------------------------------------------------
CancelledError                            Traceback (most recent call last)
<ipython-input-26-007d59320aa9> in <module>
----> 1 f.result()

CancelledError: 

I didn't realize that for the BlockingRunner the Futures never get canceled. I will have to think a bit about why that happens. The AsyncRunner's behavior is fine.