Skip to content
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
2e5c6a6
Move `ensdb` module contents from ENSNode SDK to ENSDb SDK
tk-o Mar 18, 2026
d3329fd
Add `EnsNodeDbMigrations` interface to ENSDb SDK
tk-o Mar 18, 2026
a844af0
Introduce ENSDb Reader
tk-o Mar 19, 2026
316fe67
Introduce ENSDb Writer
tk-o Mar 19, 2026
90ff103
Setup exports fro ENSDb client module
tk-o Mar 19, 2026
72e1f9b
Replace ENSDb Client implementation in ENSIndexer
tk-o Mar 19, 2026
2df0ab2
Make ENSIndexer to execute ENSNode Schema migrations in ENSDb
tk-o Mar 19, 2026
8357285
Update tests for ENSDb Writer Worker
tk-o Mar 19, 2026
b66c443
docs(changeset): Moved `ensdb` module from ENSNode SDK into ENSDb SDK.
tk-o Mar 19, 2026
187a3b5
docs(changeset): Introduced two client implementations for ENSDb: `En…
tk-o Mar 19, 2026
bcc1784
docs(changeset): Replaced a bespoke `EnsDbClient` implementation with…
tk-o Mar 19, 2026
6c82714
docs(changeset): Added running database migrations for ENSDb as a res…
tk-o Mar 19, 2026
8edabdd
Code cleanup
tk-o Mar 19, 2026
24f2d1b
Introduce testing suite to ENSDb SDK
tk-o Mar 19, 2026
45eea13
Merge remote-tracking branch 'origin/main' into feat/integrate-ensdb-…
tk-o Mar 19, 2026
4b20c50
Fix typo
tk-o Mar 19, 2026
4e216f4
Apply AI PR feedback
tk-o Mar 19, 2026
3b3882b
Merge remote-tracking branch 'origin/main' into feat/integrate-ensdb-…
tk-o Mar 19, 2026
83cf6cf
Apply suggestions from code review
tk-o Mar 20, 2026
f03192e
Apply PR feedback
tk-o Mar 20, 2026
f0c7f1f
Improve layers of responsibility in ENSDb Drizzle Client file
tk-o Mar 20, 2026
54301fc
Improve layers of responsibility in ENSDb SDK clients
tk-o Mar 20, 2026
a3dbb6b
Update testing suite for ENSDb SDK
tk-o Mar 20, 2026
9eeb935
Update docs in `ponder.schema.ts` file
tk-o Mar 20, 2026
0534ef3
Update packages/ensdb-sdk/src/client/ensdb-reader.test.ts
tk-o Mar 20, 2026
132eca8
Rename `ensindexer` file to `ensindexer-abstract` in ENSDb SDK
tk-o Mar 20, 2026
e527292
Improve the method for cloning drizzle table objects
tk-o Mar 20, 2026
121efd5
Allow using individual ENSDb Schema defintions for Drizzle querires
tk-o Mar 20, 2026
6145e50
Update test files
tk-o Mar 20, 2026
d5f7c3f
Handle database migrations execution failure
tk-o Mar 20, 2026
c5397bc
Rename `ensDbWriter` consts to `ensDbClient`
tk-o Mar 21, 2026
5ac0060
Make usage of ENSIndexer Schema explicitly "concrete"
tk-o Mar 21, 2026
593cbb8
Make `drizzle` module abstractions from ENSDb SDK not to be shared pu…
tk-o Mar 21, 2026
ba48daa
Make `EnsDbReader` constructor to build Drizzle client
tk-o Mar 21, 2026
0adcd14
Update test files
tk-o Mar 21, 2026
219b7bd
fix npm audit issues
tk-o Mar 21, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion apps/ensindexer/ponder/ponder.schema.ts
Original file line number Diff line number Diff line change
@@ -1,2 +1,10 @@
// export database schema definition for ENSIndexer
/**
* Export database schema definition for ENSIndexer
* Note: Ponder uses `globalThis.PONDER_NAMESPACE_BUILD.schema` value to
* dynamically build the "concrete" ENSIndexer Schema definition
* from the "abstract" ENSIndexer Schema definition for Ponder app to use.
Comment thread
tk-o marked this conversation as resolved.
Outdated
Comment thread
tk-o marked this conversation as resolved.
Outdated
*
* @see https://github.com/ponder-sh/ponder/blob/c8f6935fb65176c01b40cae9056be704c0e5318e/packages/core/src/build/index.ts#L380-L424
* @see https://github.com/ponder-sh/ponder/blob/6fcc15d4234e43862cb6e21c05f3c57f4c2f7464/packages/core/src/drizzle/onchain.ts#L280-L281
**/
export * from "@ensnode/ensdb-sdk/ensindexer";
Comment thread
tk-o marked this conversation as resolved.
Outdated
6 changes: 3 additions & 3 deletions apps/ensindexer/ponder/src/api/handlers/ensnode-api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,13 @@ import {
serializeIndexingStatusResponse,
} from "@ensnode/ensnode-sdk";

import { ensDbClient } from "@/lib/ensdb/singleton";
import { ensDbWriter } from "@/lib/ensdb/singleton";
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i realize that the writer inherits reader, but it still reads oddly to see ensDbWriter.get* — maybe we can just call any instance of a reader (or writer) an ensDbClient so operations read more clearly? up to you, as the current naming is technically more precise.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried ensDbClient before, but got feedback that is should be simply ensDbWriter feels a bit weird to read something from the writer, but since the writer class extends the reader class, it's technically correct.

Alternative approach would be for the writer class to use the reader class for internal reads. So it'd be composition over inheritance. In that case, we'd have two objects: ensDbReader for all reads, and ensDbWriter for all writes.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shrugs Agreed with your feedback here.

@tk-o I'm trying to remember what the motivation was for us to split the reader / writer into different classes? Was it because of something related to how only the writer needs to do database migrations? But I think the database migrations is already extracted out of the class now?

If there's a nice way to merge the reader / writer into the same class then I'm happy to do it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lightwalker-eth the original motivation was described in this slack message.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could just update the const name that is exported from @/lib/ensdb/singleton. It should be simply ensDbClient. In this case, it will be a client that can read from ENSDb and write into ENSDb.

ensDbClient will be also a useful name for ENSDb Reader instance when we integrate ENSDb SDK into ENSApi.

How does that sound?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds good 👍


const app = new Hono();

// include ENSIndexer Public Config endpoint
app.get("/config", async (c) => {
const publicConfig = await ensDbClient.getEnsIndexerPublicConfig();
const publicConfig = await ensDbWriter.getEnsIndexerPublicConfig();

// Invariant: the public config is guaranteed to be available in ENSDb after
// application startup.
Expand All @@ -30,7 +30,7 @@ app.get("/config", async (c) => {

app.get("/indexing-status", async (c) => {
try {
const crossChainSnapshot = await ensDbClient.getIndexingStatusSnapshot();
const crossChainSnapshot = await ensDbWriter.getIndexingStatusSnapshot();

// Invariant: the Indexing Status Snapshot is expected to be available in
// ENSDb shortly after application startup. There is a possibility that
Expand Down
5 changes: 2 additions & 3 deletions apps/ensindexer/ponder/src/api/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,14 @@ import { cors } from "hono/cors";

import type { ErrorResponse } from "@ensnode/ensnode-sdk";

import { migrateEnsNodeDb } from "@/lib/ensdb/migrate-ensnode-db";
import { ensDbClient } from "@/lib/ensdb/singleton";
import { migrateEnsNodeSchema } from "@/lib/ensdb/migrate-ensnode-schema";
import { startEnsDbWriterWorker } from "@/lib/ensdb-writer-worker/singleton";

import ensNodeApi from "./handlers/ensnode-api";

// Before starting the ENSDb Writer Worker, we need to ensure that
// the ENSNode Schema in ENSDb is up to date by running any pending migrations.
migrateEnsNodeDb(ensDbClient).then(() => {
migrateEnsNodeSchema().then(() => {
// The entry point for the ENSDb Writer Worker. It must be placed inside
Comment thread
tk-o marked this conversation as resolved.
Outdated
// the `api` directory of the Ponder app to avoid the following build issue:
// Error: Invalid dependency graph. Config, schema, and indexing function files
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { vi } from "vitest";

import type { EnsNodeDbMutations, EnsNodeDbQueries } from "@ensnode/ensdb-sdk";
import type { EnsDbWriter } from "@ensnode/ensdb-sdk";
import {
type CrossChainIndexingStatusSnapshot,
CrossChainIndexingStrategyIds,
Expand All @@ -16,9 +16,6 @@ import {
import type { IndexingStatusBuilder } from "@/lib/indexing-status-builder";
import type { PublicConfigBuilder } from "@/lib/public-config-builder";

// Helper type for the combined client interface used by EnsDbWriterWorker
type EnsDbClientForWorker = EnsNodeDbMutations & EnsNodeDbQueries;

// Test fixture for EnsRainbowPublicConfig
export const mockEnsRainbowPublicConfig: EnsRainbowPublicConfig = {
version: "1.0.0",
Expand Down Expand Up @@ -48,16 +45,16 @@ export const mockPublicConfig: EnsIndexerPublicConfig = {
};

// Helper to create mock objects with consistent typing
export function createMockEnsDbClient(
overrides: Partial<ReturnType<typeof baseEnsDbClient>> = {},
): EnsDbClientForWorker {
export function createMockEnsDbWriter(
overrides: Partial<ReturnType<typeof baseEnsDbWriter>> = {},
): EnsDbWriter {
return {
...baseEnsDbClient(),
...baseEnsDbWriter(),
...overrides,
} as unknown as EnsDbClientForWorker;
} as unknown as EnsDbWriter;
}
Comment thread
tk-o marked this conversation as resolved.

export function baseEnsDbClient() {
export function baseEnsDbWriter() {
return {
getEnsDbVersion: vi.fn().mockResolvedValue(undefined),
getEnsIndexerPublicConfig: vi.fn().mockResolvedValue(undefined),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import type { PublicConfigBuilder } from "@/lib/public-config-builder/public-con

import {
createMockCrossChainSnapshot,
createMockEnsDbClient,
createMockEnsDbWriter,
createMockIndexingStatusBuilder,
createMockOmnichainSnapshot,
createMockPublicConfigBuilder,
Expand Down Expand Up @@ -50,26 +50,26 @@ describe("EnsDbWriterWorker", () => {
const snapshot = createMockCrossChainSnapshot({ omnichainSnapshot });
vi.mocked(buildCrossChainIndexingStatusSnapshotOmnichain).mockReturnValue(snapshot);

const ensDbClient = createMockEnsDbClient();
const ensDbWriter = createMockEnsDbWriter();
const publicConfigBuilder = createMockPublicConfigBuilder();
const indexingStatusBuilder = createMockIndexingStatusBuilder(omnichainSnapshot);

const worker = new EnsDbWriterWorker(ensDbClient, publicConfigBuilder, indexingStatusBuilder);
const worker = new EnsDbWriterWorker(ensDbWriter, publicConfigBuilder, indexingStatusBuilder);

// act
await worker.run();

// assert - verify initial upserts happened
expect(ensDbClient.upsertEnsDbVersion).toHaveBeenCalledWith(
expect(ensDbWriter.upsertEnsDbVersion).toHaveBeenCalledWith(
mockPublicConfig.versionInfo.ensDb,
);
expect(ensDbClient.upsertEnsIndexerPublicConfig).toHaveBeenCalledWith(mockPublicConfig);
expect(ensDbWriter.upsertEnsIndexerPublicConfig).toHaveBeenCalledWith(mockPublicConfig);

// advance time to trigger interval
await vi.advanceTimersByTimeAsync(1000);

// assert - snapshot should be upserted
expect(ensDbClient.upsertIndexingStatusSnapshot).toHaveBeenCalledWith(snapshot);
expect(ensDbWriter.upsertIndexingStatusSnapshot).toHaveBeenCalledWith(snapshot);
expect(buildCrossChainIndexingStatusSnapshotOmnichain).toHaveBeenCalledWith(
omnichainSnapshot,
expect.any(Number),
Expand All @@ -86,26 +86,26 @@ describe("EnsDbWriterWorker", () => {
throw incompatibleError;
});

const ensDbClient = createMockEnsDbClient({
const ensDbWriter = createMockEnsDbWriter({
getEnsIndexerPublicConfig: vi.fn().mockResolvedValue(mockPublicConfig),
});
const publicConfigBuilder = createMockPublicConfigBuilder(mockPublicConfig);
const indexingStatusBuilder = createMockIndexingStatusBuilder();

const worker = new EnsDbWriterWorker(ensDbClient, publicConfigBuilder, indexingStatusBuilder);
const worker = new EnsDbWriterWorker(ensDbWriter, publicConfigBuilder, indexingStatusBuilder);

// act & assert
await expect(worker.run()).rejects.toThrow("incompatible");
expect(ensDbClient.upsertEnsDbVersion).not.toHaveBeenCalled();
expect(ensDbWriter.upsertEnsDbVersion).not.toHaveBeenCalled();
});

it("throws error when worker is already running", async () => {
// arrange
const ensDbClient = createMockEnsDbClient();
const ensDbWriter = createMockEnsDbWriter();
const publicConfigBuilder = createMockPublicConfigBuilder();
const indexingStatusBuilder = createMockIndexingStatusBuilder();

const worker = new EnsDbWriterWorker(ensDbClient, publicConfigBuilder, indexingStatusBuilder);
const worker = new EnsDbWriterWorker(ensDbWriter, publicConfigBuilder, indexingStatusBuilder);

// act - first run
await worker.run();
Expand All @@ -120,34 +120,34 @@ describe("EnsDbWriterWorker", () => {
it("throws error when config fetch fails", async () => {
// arrange
const networkError = new Error("Network failure");
const ensDbClient = createMockEnsDbClient();
const ensDbWriter = createMockEnsDbWriter();
const publicConfigBuilder = {
getPublicConfig: vi.fn().mockRejectedValue(networkError),
} as unknown as PublicConfigBuilder;
const indexingStatusBuilder = createMockIndexingStatusBuilder();

const worker = new EnsDbWriterWorker(ensDbClient, publicConfigBuilder, indexingStatusBuilder);
const worker = new EnsDbWriterWorker(ensDbWriter, publicConfigBuilder, indexingStatusBuilder);

// act & assert
await expect(worker.run()).rejects.toThrow("Network failure");
expect(publicConfigBuilder.getPublicConfig).toHaveBeenCalledTimes(1);
expect(ensDbClient.upsertEnsDbVersion).not.toHaveBeenCalled();
expect(ensDbWriter.upsertEnsDbVersion).not.toHaveBeenCalled();
});

it("throws error when stored config fetch fails", async () => {
// arrange
const dbError = new Error("Database connection lost");
const ensDbClient = createMockEnsDbClient({
const ensDbWriter = createMockEnsDbWriter({
getEnsIndexerPublicConfig: vi.fn().mockRejectedValue(dbError),
});
const publicConfigBuilder = createMockPublicConfigBuilder();
const indexingStatusBuilder = createMockIndexingStatusBuilder();

const worker = new EnsDbWriterWorker(ensDbClient, publicConfigBuilder, indexingStatusBuilder);
const worker = new EnsDbWriterWorker(ensDbWriter, publicConfigBuilder, indexingStatusBuilder);

// act & assert
await expect(worker.run()).rejects.toThrow("Database connection lost");
expect(ensDbClient.upsertEnsDbVersion).not.toHaveBeenCalled();
expect(ensDbWriter.upsertEnsDbVersion).not.toHaveBeenCalled();
});

it("fetches stored and in-memory configs concurrently", async () => {
Expand All @@ -156,19 +156,19 @@ describe("EnsDbWriterWorker", () => {
// validation passes
});

const ensDbClient = createMockEnsDbClient({
const ensDbWriter = createMockEnsDbWriter({
getEnsIndexerPublicConfig: vi.fn().mockResolvedValue(mockPublicConfig),
});
const publicConfigBuilder = createMockPublicConfigBuilder(mockPublicConfig);
const indexingStatusBuilder = createMockIndexingStatusBuilder();

const worker = new EnsDbWriterWorker(ensDbClient, publicConfigBuilder, indexingStatusBuilder);
const worker = new EnsDbWriterWorker(ensDbWriter, publicConfigBuilder, indexingStatusBuilder);

// act
await worker.run();

// assert - both should have been called (concurrent execution via Promise.all)
expect(ensDbClient.getEnsIndexerPublicConfig).toHaveBeenCalledTimes(1);
expect(ensDbWriter.getEnsIndexerPublicConfig).toHaveBeenCalledTimes(1);
expect(publicConfigBuilder.getPublicConfig).toHaveBeenCalledTimes(1);

// cleanup
Expand All @@ -180,18 +180,18 @@ describe("EnsDbWriterWorker", () => {
const snapshot = createMockCrossChainSnapshot();
vi.mocked(buildCrossChainIndexingStatusSnapshotOmnichain).mockReturnValue(snapshot);

const ensDbClient = createMockEnsDbClient();
const ensDbWriter = createMockEnsDbWriter();
const publicConfigBuilder = createMockPublicConfigBuilder();
const indexingStatusBuilder = createMockIndexingStatusBuilder();

const worker = new EnsDbWriterWorker(ensDbClient, publicConfigBuilder, indexingStatusBuilder);
const worker = new EnsDbWriterWorker(ensDbWriter, publicConfigBuilder, indexingStatusBuilder);

// act
await worker.run();

// assert - config should be called once (pRetry is mocked)
expect(publicConfigBuilder.getPublicConfig).toHaveBeenCalledTimes(1);
expect(ensDbClient.upsertEnsIndexerPublicConfig).toHaveBeenCalledWith(mockPublicConfig);
expect(ensDbWriter.upsertEnsIndexerPublicConfig).toHaveBeenCalledWith(mockPublicConfig);

// cleanup
worker.stop();
Expand All @@ -202,11 +202,11 @@ describe("EnsDbWriterWorker", () => {
it("stops the interval when stop() is called", async () => {
// arrange
const upsertIndexingStatusSnapshot = vi.fn().mockResolvedValue(undefined);
const ensDbClient = createMockEnsDbClient({ upsertIndexingStatusSnapshot });
const ensDbWriter = createMockEnsDbWriter({ upsertIndexingStatusSnapshot });
const publicConfigBuilder = createMockPublicConfigBuilder();
const indexingStatusBuilder = createMockIndexingStatusBuilder();

const worker = new EnsDbWriterWorker(ensDbClient, publicConfigBuilder, indexingStatusBuilder);
const worker = new EnsDbWriterWorker(ensDbWriter, publicConfigBuilder, indexingStatusBuilder);

// act
await worker.run();
Expand All @@ -227,11 +227,11 @@ describe("EnsDbWriterWorker", () => {
describe("isRunning - worker state", () => {
it("indicates isRunning status correctly", async () => {
// arrange
const ensDbClient = createMockEnsDbClient();
const ensDbWriter = createMockEnsDbWriter();
const publicConfigBuilder = createMockPublicConfigBuilder();
const indexingStatusBuilder = createMockIndexingStatusBuilder();

const worker = new EnsDbWriterWorker(ensDbClient, publicConfigBuilder, indexingStatusBuilder);
const worker = new EnsDbWriterWorker(ensDbWriter, publicConfigBuilder, indexingStatusBuilder);

// assert - not running initially
expect(worker.isRunning).toBe(false);
Expand Down Expand Up @@ -267,7 +267,7 @@ describe("EnsDbWriterWorker", () => {

vi.mocked(buildCrossChainIndexingStatusSnapshotOmnichain).mockReturnValue(crossChainSnapshot);

const ensDbClient = createMockEnsDbClient();
const ensDbWriter = createMockEnsDbWriter();
const publicConfigBuilder = createMockPublicConfigBuilder();
const indexingStatusBuilder = {
getOmnichainIndexingStatusSnapshot: vi
Expand All @@ -276,7 +276,7 @@ describe("EnsDbWriterWorker", () => {
.mockResolvedValueOnce(validSnapshot),
} as unknown as IndexingStatusBuilder;

const worker = new EnsDbWriterWorker(ensDbClient, publicConfigBuilder, indexingStatusBuilder);
const worker = new EnsDbWriterWorker(ensDbWriter, publicConfigBuilder, indexingStatusBuilder);

// act - run returns immediately
await worker.run();
Expand All @@ -289,8 +289,8 @@ describe("EnsDbWriterWorker", () => {

// assert
expect(indexingStatusBuilder.getOmnichainIndexingStatusSnapshot).toHaveBeenCalledTimes(2);
expect(ensDbClient.upsertIndexingStatusSnapshot).toHaveBeenCalledTimes(1);
expect(ensDbClient.upsertIndexingStatusSnapshot).toHaveBeenCalledWith(crossChainSnapshot);
expect(ensDbWriter.upsertIndexingStatusSnapshot).toHaveBeenCalledTimes(1);
expect(ensDbWriter.upsertIndexingStatusSnapshot).toHaveBeenCalledWith(crossChainSnapshot);

// cleanup
worker.stop();
Expand All @@ -317,7 +317,7 @@ describe("EnsDbWriterWorker", () => {
.mockReturnValueOnce(crossChainSnapshot2)
.mockReturnValueOnce(crossChainSnapshot2);

const ensDbClient = createMockEnsDbClient({
const ensDbWriter = createMockEnsDbWriter({
upsertIndexingStatusSnapshot: vi
.fn()
.mockResolvedValueOnce(undefined)
Expand All @@ -333,24 +333,24 @@ describe("EnsDbWriterWorker", () => {
.mockResolvedValueOnce(snapshot2),
} as unknown as IndexingStatusBuilder;

const worker = new EnsDbWriterWorker(ensDbClient, publicConfigBuilder, indexingStatusBuilder);
const worker = new EnsDbWriterWorker(ensDbWriter, publicConfigBuilder, indexingStatusBuilder);

// act
await worker.run();

// first tick - succeeds
await vi.advanceTimersByTimeAsync(1000);
expect(ensDbClient.upsertIndexingStatusSnapshot).toHaveBeenCalledWith(crossChainSnapshot1);
expect(ensDbWriter.upsertIndexingStatusSnapshot).toHaveBeenCalledWith(crossChainSnapshot1);

// second tick - fails with DB error, but continues
await vi.advanceTimersByTimeAsync(1000);
expect(ensDbClient.upsertIndexingStatusSnapshot).toHaveBeenLastCalledWith(
expect(ensDbWriter.upsertIndexingStatusSnapshot).toHaveBeenLastCalledWith(
crossChainSnapshot2,
);

// third tick - succeeds again
await vi.advanceTimersByTimeAsync(1000);
expect(ensDbClient.upsertIndexingStatusSnapshot).toHaveBeenCalledTimes(3);
expect(ensDbWriter.upsertIndexingStatusSnapshot).toHaveBeenCalledTimes(3);

// cleanup
worker.stop();
Expand Down
Loading
Loading