Skip to content

mcp-standalone tests flake on slow CI — make livez probe injectable #449

Closed

Description

@rohitg00

Context

test/mcp-standalone.test.ts is the only consistently flaky test file in the suite. Symptom: 1-11 tests fail intermittently, all in this file, with logs like:

[@agentmemory/mcp] livez probe http://localhost:3111/agentmemory/livez failed in 2000ms: ECONNREFUSED;
falling back to local InMemoryKV

The shim tries to probe a real :3111 during test runs. On a fast machine the fallback fires within the test budget and tests pass. On a loaded machine (or in parallel with other vitest workers), the probe times out + the fallback timing varies + assertions about call-count or response shape go non-deterministic.

This has been the pre-existing 10-11 "expected failures" we've been quoting across the last 5 releases.

Proposal

Inject the probe transport. Make livezProbe a constructor-time dependency on the shim's main module so tests can pass a stub that returns {ok: false} instantly without hitting the network.

// src/mcp/standalone.ts
export interface ShimDeps {
  livezProbe?: (url: string, timeoutMs: number) => Promise<Response>;
}

export function createShim(deps: ShimDeps = {}) {
  const probe = deps.livezProbe ?? defaultLivezProbe;
  // ... rest of shim wiring uses probe(...) instead of bare fetch
}

Tests pass a stub probe that fails fast. Production keeps the default fetch-based probe.

Files

  • src/mcp/standalone.ts — extract livezProbe into a DI seam
  • test/mcp-standalone.test.ts — stub the probe in each describe block

Acceptance

  • npm test -- --reporter=verbose test/mcp-standalone.test.ts runs 10 times in a row without a single failure
  • No real network calls during the test run (verified by mocking global.fetch and asserting zero calls in the probe path)

Why this matters

We've been documenting "10-11 pre-existing failures unrelated to this PR" in every release for two months. Stable test suite makes CI signal trustworthy and unblocks gating-on-green for the merge train.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions