Skip to content

/autoplan aggregator silently returns 0 tasks: jq scope bug in commit filter #2018

Closed

Description

@jcilley-mhvfcu

Summary

/autoplan's Phase 4 Implementation Tasks aggregator returns zero tasks for every invocation because of a jq scope bug. The error is swallowed by 2>/dev/null, so users see an empty aggregated list with no diagnostic, even when the per-phase tasks-<phase>-*.jsonl files contain valid records on the current branch/commit.

Repro'd against v1.58.1.0 (also affects v1.52.2.0).

Location

autoplan/SKILL.md:1641 (current main):

select(.branch == $branch and ($commits | split("|") | index(.commit) != null))

Root cause

Inside index(.commit), .commit is evaluated against jq's current input, which at that point is the array produced by $commits | split("|") — NOT the outer record. jq tries to "index the split array by the string commit" and errors out:

jq: error (at <stdin>:1): Cannot index array with string "commit"

The error is suppressed by 2>/dev/null on the surrounding jq ... >> "$ALL_JSONL" invocation, so the aggregator silently drops every record and the user sees:

No actionable tasks emitted from any phase.

…even when the JSONL files clearly contain matching records on the current branch and recent commits.

Repro

echo '{"branch":"dev","commit":"abc123","phase":"ceo-review"}' | \
  jq --arg branch "dev" --arg commits "abc123|def456" \
  'select(.branch == $branch and ($commits | split("|") | index(.commit) != null))'
# jq: error (at <stdin>:1): Cannot index array with string "commit"

Triggered today against MHVFCU/mhv-member-manager with two valid JSONL files (12 CEO tasks + 20 Eng tasks, both on dev, both with commit values present in the last-5-commits filter set). Aggregator output: 0 tasks. After applying the fix below: 32 tasks rendered correctly.

Fix

Bind the record before the inner pipeline so .commit resolves to the record's field, not the split array:

. as $r | select($r.branch == $branch and ($commits | split("|") | index($r.commit) != null))

Equivalent alternative using IN/1:

select(.branch == $branch and (.commit | IN($commits | split("|") | .[])))

Both verified working against the same input.

Suggested patch

Diff against autoplan/SKILL.md:1641:

-      'select(.branch == $branch and ($commits | split("|") | index(.commit) != null))' \
+      '. as $r | select($r.branch == $branch and ($commits | split("|") | index($r.commit) != null))' \

Why this matters

Every /autoplan run is currently shipping an empty ### Implementation Tasks (aggregated across phases) section at the Final Approval Gate. Users either:

  1. Don't notice (and lose the aggregated checklist value that /autoplan promises), or
  2. Notice and have to re-aggregate by hand.

The 2>/dev/null masks the actual jq error from anyone running with verbose shells, which makes this near-impossible to diagnose without dropping the redirect.

Related: defensive improvement

While you're in there, consider dropping the 2>/dev/null on the per-file jq invocation OR teeing stderr to a debug file. The current silent-error pattern hides not just this bug but any future malformed-JSONL case.

Repro environment: macOS 15.5, jq 1.7.1, bash 5.x, zsh 5.x. Bug is jq-version-independent (it's a scope/parsing issue, not a version-specific behavior change).

Activity

  1. 0xDevNinja commented on Jun 16, 2026

    @0xDevNinja
    Contributor

    Picking this up. The root cause is exactly as described: inside ($commits | split("|") | index(.commit)) the pipe input is the split array, so index(.commit) indexes the array with the string "commit" and jq errors Cannot index array with string "commit", swallowed by the surrounding 2>/dev/null. Fix binds the record's commit to a variable before entering the split pipe so index() gets the commit string. Opening a PR with a free regression test that runs the shipped filter through jq.

  2. optimization2026 commented on Jun 30, 2026

    @optimization2026

    This is an output contract problem in /autoplan.

    A plan aggregator should not silently return zero tasks when upstream phase output exists.

    The contract should require:

    phase outputs exist
      -> aggregator parses them
      -> task count > 0 unless plan intentionally has no work
      -> if task count is 0, emit diagnostic and block success
    

    A standard AISP-style contract could express this:

    [
      {
        "role": "system",
        "content": {
          "protocol": "AISP V1.0.0",
          "axiom_0": "Human_Sovereignty_and_Wellbeing",
          "id": "gstack_autoplan_task_aggregation_contract_aisp",
          "name": "gstack /autoplan Task Aggregation Contract",
          "version": "1.0.0",
          "license": "MIT",
          "summary": "Machine-checkable contract preventing /autoplan from silently producing zero tasks after non-empty phase output.",
          "flow_format": "mermaid",
          "loading_mode": "node",
          "tools": ["filesystem", "shell"],
          "params": {
            "plan_dir": "string",
            "phase_outputs_path": "string"
          },
          "system_prompt": ""
        }
      },
      {
        "role": "user",
        "content": {
          "instruction": "STRICTLY OBEY aisp_contract; its non_negotiable rules are inviolable; then RUN aisop.main",
          "user_input": "{user_input}",
          "aisp_contract": {
            "profile": "aisp.skill.v1",
            "invocation": {
              "mode": "autoplan_validation",
              "when_to_use": [
                "after /autoplan review phases complete",
                "before writing final task list",
                "when aggregator output is zero tasks"
              ],
              "when_not_to_use": [
                "explicit no-op plan approved by user",
                "dry-run phase with no planned implementation"
              ]
            },
            "non_negotiable": [
              {
                "rule": "If upstream phase outputs contain implementation work, final task count must not be zero.",
                "enforced_by": "validate.step2:sys.assert"
              },
              {
                "rule": "Zero-task output must include a diagnostic reason.",
                "enforced_by": "diagnose.step2:sys.assert"
              }
            ],
            "discovery": {
              "category": "planning_workflow",
              "tags": ["gstack", "autoplan", "tasks", "aggregation", "jq", "aisp"]
            },
            "risk_level": "medium",
            "resources": [
              {
                "id": "phase_outputs",
                "path": "{phase_outputs_path}",
                "kind": "artifact",
                "mode": "read_only",
                "when": "Read before final task aggregation.",
                "scope": "skill"
              }
            ]
          },
          "aisop": {
            "main": "graph TD\n    load[Load phase outputs] --> aggregate[Aggregate tasks]\n    aggregate --> validate[Validate task count]\n    validate --> diagnose[Diagnose zero-task case]\n    diagnose --> emit[Emit task list or error]\n    emit --> end_node((End))"
          },
          "functions": {
            "load": {
              "step1": "sys.io.read(phase_outputs_path) -> phase_outputs",
              "output_mapping": "phase_outputs"
            },
            "aggregate": {
              "step1": "Aggregate tasks from phase_outputs.",
              "output_mapping": "aggregated_tasks"
            },
            "validate": {
              "step1": "Check whether phase_outputs imply implementation work.",
              "step2": "sys.assert('task count > 0 when implementation work exists', 'Autoplan produced zero tasks despite upstream work')",
              "output_mapping": "task_count_validation"
            },
            "diagnose": {
              "step1": "If task count is zero, report parser filter, jq scope, empty input, or intentional no-op.",
              "step2": "sys.assert('zero task case has diagnostic reason', 'Zero-task output requires diagnostic')",
              "output_mapping": "zero_task_diagnostic"
            },
            "emit": {
              "step1": "Emit validated task list or actionable error.",
              "output_mapping": "task_output"
            },
            "end_node": {
              "step1": "Return task_output."
            }
          }
        }
      }
    ]

    The practical fix:

    1. Fix the jq scope bug.
    2. Add a guard: non-empty upstream implementation findings cannot produce empty task list.
    3. Add diagnostic output for zero-task cases.
    4. Add regression fixture for the jq filter.
    
  3. added 2 commits that reference this issue on Jul 14, 2026
    703e202
    a1997cb
  4. time-attack commented on Jul 15, 2026

    @time-attack
    Contributor

    @16francej — recommend closing this issue only after #2021 lands with its required checks accepted. The current filter rebinds its input before reading the commit and turns the resulting failure into zero tasks. Keep broader zero-task and malformed-input diagnostics as a separate follow-up.

  5. added a commit that references this issue on Aug 15, 2026
    008dd65
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions