openai_model doesn't accept `assistant` role messages

[cwing-nvidia]

Describe the bug

The OpenAI wrapper for APIs doesn't accept some 'assistant' role messages in the conversation history. This blocks multi-turn implementations where the agent's previous response was included as an 'assistant' message.

TALES typically represents the observations as 'user' inputs and the actions as 'assistant' inputs. NeMo throws an error when there are 'assistant' tagged messages so in the rollouts we just use the 'developer' tag for now.

Current workaround:

# During execution, use "developer" role instead of "assistant" to avoid error
message_history.append({"role": "developer", "content": action})

# Later, convert to proper format for saving/training
corrected_message_history = [message_history[0]]
for turn in message_history[1:]:
    if turn["role"] == "developer":
        # Convert developer → assistant
        corrected_message_history.append({"role": "assistant", "content": turn["content"]})
    else:
        corrected_message_history.append(turn)

Steps/Code to reproduce bug

Input

{
  "messages": [
    {
      "role": "developer", 
      "content": "You are playing a text-based game..."
    },
    {
      "role": "user", 
      "content": "You are in a kitchen..."
    },
    {
      "role": "assistant",
      "content": "read cookbook"
    },
    {
      "role": "user", 
      "content": "You open the cookbook..."
    }
  ]
}

Expected output

{\"id\": \"chatcmpl-CR4cIxdAB1rQZmrOkeZKMEL5WlfOA\", \"choices\": [{\"finish_reason\": \"stop\", \"index\": 0, \"logprobs\": null, \"message\": {\"content\": \"open fridge\", \"refusal\": null, \"role\": \"assistant\", \"annotations\": [], \"audio\": null, \"function_call\": null, \"tool_calls\": null}}], \"created\": 1760568638, \"model\": \"gpt-4.1-2025-04-14\", \"object\": \"chat.completion\", \"service_tier\": \"default\", \"system_fingerprint\": \"fp_422e2d36a8\", \"usage\": {\"completion_tokens\": 2, \"prompt_tokens\": 305, \"total_tokens\": 307, \"completion_tokens_details\": {\"accepted_prediction_tokens\": 0, \"audio_tokens\": 0, \"reasoning_tokens\": 0, \"rejected_prediction_tokens\": 0}, \"prompt_tokens_details\": {\"audio_tokens\": 0, \"cached_tokens\": 0}}}", "reward": 0}

Error

{
  "detail": [
    {
      "type": "literal_error",
      "loc": ["body", "messages", 2, "NeMoGymChatCompletionDeveloperMessageParam", "role"],
      "msg": "Input should be 'developer'",
      "input": "assistant",
      "ctx": {"expected": "'developer'"}
    },
    {
      "type": "missing",
      "loc": ["body", "messages", 2, "NeMoGymChatCompletionAssistantMessageParam", "tool_calls"],
      "msg": "Field required",
      "input": {"role": "assistant", "content": "read cookbook"}
    },
    ...
  ]
}

Expected behavior

OpenAI wrapper should accept standard OpenAI message format including 'assistant' role messages, even if tool_calls were not

Environment details

TALES

Additional context

@christopherzc mentioned in PR: #203

[cwing-nvidia]

Bug analysis:

The error shows that NeMoGymChatCompletionAssistantMessageParam requires tool_calls field even when the assistant message is just text content (no tool use). This is stricter than standard OpenAI API, which allows assistant messages with just content.

In nemo_gym/openai_utils.py

class NeMoGymChatCompletionAssistantMessageParam(ChatCompletionAssistantMessageParam):
    # Override the iterable which is annoying to work with.
    content: Union[str, List[ContentArrayOfContentPart], None]
    tool_calls: List[NeMoGymChatCompletionMessageToolCallParam]  # <-- Missing Optional

[cwing-nvidia]

I think this issue was already fixed in PR #234 (commit 40bf7d4) by @bxyu-nvidia on Oct 24, 2025.

The fix made tool_calls optional in two ways:

1. Added total=False to the class definition
2. Changed the field to Optional[List[...]] = None

Assistant messages with just content (no tool_calls) should now work correctly.

@christopherzc the latest main branch should resolve this issue, can you check?

[christopherzc]

Just tested locally and am having no issues with assistant messages after updating to the current main! I'll update the PR, remove the bug replication script, and push things in a bit!