Skip to content

post_rollout can crash + does not have completion rendered #726

@mikasenghaas

Description

@mikasenghaas

2 issues with the current way post_rollout is set up:

  • it does not have a try..except so can crash a run if not handled very carefully by the env designer. should handle this more holistically probably
 File "/home/ubuntu/research-environments/environments/code_env/code_env/code_env.py", line 229, in post_rollout
    completion = state["trajectory"][-1]["completion"]
                 ~~~~~~~~~~~~~~~~~~~^^^^
IndexError: list index out of range
  • it does not have the completion field set in state yet because it gets called before this gets rendered -- this used to be different and is a bit unexpected given that e.g. rubric can always access the completion field

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions