Simulation Checkpointing - Long Term

[JamesArruda]

It would be very useful to be able to checkpoint a simulation. Meaning to be able to save its entire state and restart it at the exact time it was checkpointed.

The simpy mailing group brought up os.fork() (Linux-only) as a possible solution, but that isn't fully compatible with all OS-es, and there may be use cases where checkpointing may want to save to a file for later use.

Initial thoughts for what this requires:

1. Storing all state values (mostly done)
2. Storing all knowledge
3. Recording positions in tasks
4. Recording all events in Simpy's queue
5. Enforcing assumptions - such as only using UPSTAGE events, actors, and tasks. Processes or other complications likely won't work.
6. Reloading, playing through a task network to reach the correct position, setting the states, etc.

[JamesArruda]

The question to answer to help understand if this is worthwhile is: Why not just ensure deterministic to a seed and run it forward?

Thoughts:

1. Runtime to do that, depending on the sim. (What's the tradeoff for saving/loading?)
2. Redundant data recording, maybe turn it off until some point?
3. If UPSTAGE were to be a part of a co-simulation, it may need to be able to step backwards slightly - which would be assisted by many of the components of a checkpointing system
4. ...