feat(ensindex): introduce registrars plugin#1249
Conversation
🦋 Changeset detectedLatest commit: 0b4b32b The changes in this PR will be included in the next version bump. This PR includes changesets to release 14 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
521d681 to
4895125
Compare
lightwalker-eth
left a comment
There was a problem hiding this comment.
@tk-o Hey thanks for sharing this! Overall looking good. Shared a detail review and feedback mostly on terminology but also a few ideas that would influence code.
| subregistryId: t.text().primaryKey(), | ||
|
|
||
| /** | ||
| * The node of a name the subregistry manager. Example managed names: |
There was a problem hiding this comment.
| * The node of a name the subregistry manager. Example managed names: | |
| * The node (namehash) of the name the subregistry manages subnames of. Example subregistry managed names: |
| /** | ||
| * Registration Lifecycle | ||
| */ |
There was a problem hiding this comment.
| /** | |
| * Registration Lifecycle | |
| */ | |
| /** | |
| * Registration Lifecycle | |
| * | |
| * A "registration lifecycle" represents a single cycle of a name being registered once followed by renewals | |
| * (expiry date extensions) any number of times. | |
| * | |
| * Note that this data model only tracks the *most recently created* "registration lifecycle" record for a | |
| * name and doesn't track *all* "registration lifecycle" records for a name across time. Therefore, if a | |
| * name goes through multiple cycles of: | |
| * (registration -> expiry -> release) -> (registration -> expiry -> release) -> etc.. | |
| * this data model only stores data of the most recently created "registration lifecycle". | |
| * | |
| * For now we make the following simplifying assumptions: | |
| * 1. That no two subregistries hold state for the same node. | |
| * 2. That the subregistry associated with the name X in the ENS root registry exclusively holds state for subnames of X. | |
| * | |
| * These simplifying assumptions happen to be true for the scope of our current indexing logic, | |
| * but nothing in the ENS protocol fundamentally forces this to always be true. Therefore this data model | |
| * will need refactoring in the future as our indexing logic expands to handle more complex scenarios. | |
| */ |
| * The node of the FQDN of the domain this is associated with, | ||
| * guaranteed to be a subname of the associated subregistry | ||
| * for which the registration was executed. | ||
| * | ||
| * Guaranteed to be a hex string representation of 32-bytes. |
There was a problem hiding this comment.
| * The node of the FQDN of the domain this is associated with, | |
| * guaranteed to be a subname of the associated subregistry | |
| * for which the registration was executed. | |
| * | |
| * Guaranteed to be a hex string representation of 32-bytes. | |
| * The node (namehash) of the FQDN of the domain the registration lifecycle is associated with. | |
| * | |
| * Guaranteed to be a subname of the node (namehash) of the subregistry identified by `subregistryId`. | |
| * | |
| * Guaranteed to be a hex string representation of 32-bytes. |
| * Logical Subregistry Action Metadata | ||
| * | ||
| * Building a single Logical Subregistry Action requires data from multiple | ||
| * onchain events. While handling the first event, we create a temporary | ||
| * Logical Subregistry Action Metadata record where we store `logicalEventId`. | ||
| * | ||
| * The `logicalEventId` is used by subsequent event handlers to update | ||
| * the Logical Subregistry Action record. In order to get `logicalEventId`, | ||
| * an event handler creates `logicalEventKey` from the currently handled | ||
| * onchain event. | ||
| * | ||
| * The very last event handler must remove the record referenced with | ||
| * `logicalEventKey` value. |
There was a problem hiding this comment.
| * Logical Subregistry Action Metadata | |
| * | |
| * Building a single Logical Subregistry Action requires data from multiple | |
| * onchain events. While handling the first event, we create a temporary | |
| * Logical Subregistry Action Metadata record where we store `logicalEventId`. | |
| * | |
| * The `logicalEventId` is used by subsequent event handlers to update | |
| * the Logical Subregistry Action record. In order to get `logicalEventId`, | |
| * an event handler creates `logicalEventKey` from the currently handled | |
| * onchain event. | |
| * | |
| * The very last event handler must remove the record referenced with | |
| * `logicalEventKey` value. | |
| * Logical Subregistry Action Metadata | |
| * | |
| * NOTE: This table is an internal implementation detail of ENSIndexer and should not be queried outside of | |
| * ENSIndexer. | |
| * | |
| * Building a "logical subregistry action" record may require data from multiple onchain events. To help | |
| * aggregate data from multiple events into a single "logical subregistry action" ENSIndexer may temporarily | |
| * store data here to achieve this data aggregation. | |
| * | |
| * Note how multiple "logical subregistry actions" may be taken on the same `node` in the same | |
| * `transactionHash`. For example, consider a case of a single transaction registering a name and | |
| * subsequently renewing it twice. While this may be silly it is technically possible and therefore such cases | |
| * must be considered. To support such cases, when the last event handler for a "logical subregistry action" | |
| * has completed its processing the record referenced by the `logicalEventKey` must be removed. |
| * Logical Event ID | ||
| * | ||
| * A string holding the ID value to an existing Logical Registrar Action | ||
| * record that was inserted while e use this event to initiate the Logical Registrar Action record. |
There was a problem hiding this comment.
| * Logical Event ID | |
| * | |
| * A string holding the ID value to an existing Logical Registrar Action | |
| * record that was inserted while e use this event to initiate the Logical Registrar Action record. | |
| * Logical Event ID | |
| * | |
| * A string holding the `id` value of the existing "logical registrar action" record that is currently being | |
| * built as an aggregation of onchain events. | |
| * | |
| * May be used by subsequent event handlers to identify which "logical registrar action" to aggregate | |
| * additional indexed state into. |
| * | ||
| * - many RegistrationLifecycles |
There was a problem hiding this comment.
| * | |
| * - many RegistrationLifecycles | |
| * | |
| * Each Subregistry is related to: | |
| * - 0 or more RegistrationLifecycles |
| * | ||
| * - exactly one Subregistry |
There was a problem hiding this comment.
| * | |
| * - exactly one Subregistry | |
| * | |
| * Each Registration Lifecycle is related to: | |
| * - exactly one Subregistry |
| * | ||
| * - exactly one Registration Lifecycle |
There was a problem hiding this comment.
| * | |
| * - exactly one Registration Lifecycle | |
| * | |
| * Each Registrar Action is related to: | |
| * - exactly one Registration Lifecycle (note the docs on Registration Lifecycle explaining how these records may be | |
| * recycled across time). |
lightwalker-eth
left a comment
There was a problem hiding this comment.
@tk-o Hey thanks for sharing this! Overall looking good. Shared a detail review and feedback mostly on terminology but also a few ideas that would influence code.
Drop "logical" from symbol names, keep the "logical" reference in JSDocs."
Better type inference, support for bigint to number conversion.
The module includestypes and helpers supporting goals of the `registrars` plugin.
6acf439 to
f1fb889
Compare
Hosts functionality for working with dates and time.
Favour simple business logic functions. Convert types, if possible, at the edge (ponder event handlers).
registrars pluginregistrars plugin
lightwalker-eth
left a comment
There was a problem hiding this comment.
@tk-o Hey great to see these updates! I need to step away for 20 minutes but here's some quick feedback that was focused on a few other little refinements to the schema.
| @@ -0,0 +1,358 @@ | |||
| /** | |||
| * Schema Definitions for tracking of ENS subregistries. | |||
There was a problem hiding this comment.
@tk-o I think best to leave the plugin name as it is for now, but just update this comment.
| * Schema Definitions for tracking of ENS subregistries. | |
| * Schema Definitions for tracking of ENS registrars. |
| * Identifies the chainId and address of the smart contract associated | ||
| * with the subregistry. | ||
| * | ||
| * Guaranteed to be a string formatted according to the CAIP-10 standard. |
There was a problem hiding this comment.
| * Guaranteed to be a string formatted according to the CAIP-10 standard. | |
| * Guaranteed to be a fully lowercase string formatted according to the CAIP-10 standard. |
Is that fair?
| * - `base.eth` | ||
| * - `linea.eth` | ||
| * | ||
| * Guaranteed to be a hex string representation of 32-bytes. |
There was a problem hiding this comment.
| * Guaranteed to be a hex string representation of 32-bytes. | |
| * Guaranteed to be a fully lowercase hex string representation of 32-bytes. |
Is that fair?
| * Guaranteed to be a subname of the node (namehash) of the subregistry | ||
| * identified by `subregistryId`. | ||
| * | ||
| * Guaranteed to be a hex string representation of 32-bytes. |
There was a problem hiding this comment.
| * Guaranteed to be a hex string representation of 32-bytes. | |
| * Guaranteed to be a fully lowercase hex string representation of 32-bytes. |
| * Identifies the chainId and address of the subregistry smart contract | ||
| * that manages the registration lifecycle. | ||
| * | ||
| * Guaranteed to be a string formatted according to the CAIP-10 standard. |
There was a problem hiding this comment.
| * Guaranteed to be a string formatted according to the CAIP-10 standard. | |
| * Guaranteed to be a fully lowercase string formatted according to the CAIP-10 standard. |
| * - Ordered chronologically (ascending) by logIndex within `blockNumber` | ||
| * on `chainId`. |
There was a problem hiding this comment.
| * - Ordered chronologically (ascending) by logIndex within `blockNumber` | |
| * on `chainId`. | |
| * - Ordered chronologically (ascending) by logIndex within `blockNumber`. |
| * - The first element in the array is equal to the `id` of | ||
| * the "logical registrar action". |
There was a problem hiding this comment.
| * - The first element in the array is equal to the `id` of | |
| * the "logical registrar action". | |
| * - The first element in the array is equal to the `id` of | |
| * the overall "logical registrar action" record. |
| * | ||
| * Guaranteed to: | ||
| * - Reference at least one event. | ||
| * - Keep event references ordered chronologically, by event log index. |
There was a problem hiding this comment.
| * | |
| * Guaranteed to: | |
| * - Reference at least one event. | |
| * - Keep event references ordered chronologically, by event log index. |
These ideas are already documented above.
| * NOTE: This table is an internal implementation detail of ENSIndexer and | ||
| * should not be queried outside of ENSIndexer. | ||
| * | ||
| * Building a "logical subregistry action" record may require data from |
There was a problem hiding this comment.
I note that we are mixing up the following terminology:
- "logical subregistry action"
- "logical registrar action"
- "Logical" Registrar Action ID
Suggest we update all of these to one of the following depending on the context:
- "logical registrar action"
- "logical registrar action" ID
Please search through all our code and docs to refine these inconsistencies. Thanks!
| /** | ||
| * Logical Event Key | ||
| * | ||
| * A string formatted as: |
There was a problem hiding this comment.
| * A string formatted as: | |
| * A fully lowercase string formatted as: |
lightwalker-eth
left a comment
There was a problem hiding this comment.
@tk-o Hey great job 🚀 Reviewed and shared feedback. Please take the lead to merge this when ready 👍
| - direct subnames of the Lineanames registrar managed name (ex: for mainnet `linea.eth` but varies for other namespaces). | ||
|
|
||
| Additionally indexes: | ||
| - All Registrar Controllers ever associated with a known Registrar contract. |
There was a problem hiding this comment.
Should we remove this line now?
| import { durationBetween } from "./datetime"; | ||
|
|
||
| describe("datetime", () => { | ||
| describe("durationBetween()", () => { |
There was a problem hiding this comment.
Suggest to also add a unit test for the duration between being 0
| import { bigIntToNumber } from "./numbers"; | ||
|
|
||
| describe("Numbers", () => { | ||
| it("can convert bigint to number when possible", () => { |
There was a problem hiding this comment.
Can you please create another unit test demonstrating successful conversion of the smallest possible successful conversion?
| * @throws when value is too low. | ||
| * @throws when value is too high . |
There was a problem hiding this comment.
| * @throws when value is too low. | |
| * @throws when value is too high . | |
| * @throws when value is outside the range of `Number.MIN_SAFE_INTEGER` and `Number.MAX_SAFE_INTEGER`. |
| */ | ||
| export interface RegistrationLifecycle { | ||
| /** | ||
| * Subregistry account that this Registration Lifecycle belongs to. |
There was a problem hiding this comment.
| * Subregistry account that this Registration Lifecycle belongs to. | |
| * Subregistry that manages this Registration Lifecycle. |
| /** | ||
| * Block ref | ||
| * | ||
| * References the block where "logical" Registrar Action was executed. |
There was a problem hiding this comment.
| * References the block where "logical" Registrar Action was executed. | |
| * References the block where the "logical" Registrar Action was executed. |
| pricing: RegistrarActionPricing; | ||
|
|
||
| /** | ||
| * Referral information related to performing this "logical" Registrar Action. |
There was a problem hiding this comment.
| * Referral information related to performing this "logical" Registrar Action. | |
| * Referral information associated with this "logical" Registrar Action. |
| registrationLifecycle: RegistrationLifecycle; | ||
|
|
||
| /** | ||
| * Pricing information for performing this "logical" Registrar Action. |
There was a problem hiding this comment.
| * Pricing information for performing this "logical" Registrar Action. | |
| * Pricing information associated with this "logical" Registrar Action. |
| * Registration Lifecycle that this "logical" Registrar Action was | ||
| * executed for. |
There was a problem hiding this comment.
| * Registration Lifecycle that this "logical" Registrar Action was | |
| * executed for. | |
| * Registration Lifecycle associated with this "logical" Registrar Action. |
| * Account that initiated the registrarAction and is paying the "total". | ||
| * It may not be the owner of the name: | ||
| * | ||
| * 1. When a name is registered, the initial owner of the name may be |
There was a problem hiding this comment.
There's some nice ideas documented here. Can you please confirm they are also documented in the Drizzle schema?
…l registrations and renewals for direct subnames of `eth`, `base.eth`, and `linea.eth`.
Rename `RegistrarActionPricingNotApplicable` type to be `RegistrarActionPricingUnknown`
31359fb to
0b4b32b
Compare
Related to:
Replaces:
registrarsplugin #1196Resolves:
Before sharing what's been done in this PR, I'll share what's not:
registrarsplugin does not index FQDNs related to registrationssubgraphplugin andsubregistryplugin. It requires ENSIndexer to run with both plugins in order to provide data required to serve the Registrar Actions API route. This route, once added in a follow-up PR, will serve data about registartions and renewal, including FQDN for each registration.Now, about what's been achieved...
This PR introduces the
registrarsplugin.The main goal of the new plugin is to index activity in all subregistries which have ever managed names for:
We now track activity in BaseRegistrar implementation contracts with following event handlers:
NameRegisteredandNameRenewed(and their equivalents) to know about all registration updates within the subregistry.Furthermore, we track "Registrar Actions", which are
NameRegisteredandNameRenewedevents emitted by registrar controllers.Why do we need to track
NameRegisteredandNameRenewedevents emitted by different types of contracts?I'd be best to only index events from registrar controller contracts. Those events can include useful information, ie. about fees, or referrals.
However, we don't currently index all registrar controllers contracts for various reasons. For example, there were couple of special use case registrar controllers that were used for addressing security incident. We have to still learn how to best index those.
And that's where
NameRegisteredandNameRenewedevents emitted by BaseRegistrar implementation contracts are useful. We index all those contracts that ever existed, and thanks to that, we get a comprehensive view on all registrations that were ever created.One major todo item is to introduce an invariant such that we only index renewals that were made for either:
now<expiresAt).expiresAt<now<expiresAt + GRACE_PERIOD).We cannot implement this invariant at the moment, as it requires all registrar controllers to be indexed first.
TODOs
Review
Feel free to start review in the following order 👇
ENSNode Schema
packages/ensnode-schema/src/schemas/registrars.schema.tsincludes new database schemas:subregistriestracks all indexed subregistries.registrationLifecyclestracks an aggregate statte of all registrations lifecacles that were ever created in relevant BaseRegistrar implementation contract, and updated by relevant Registrar Controller contracts.registrarActiontracks "logical" actions taken by the end user, such as "registration" action, and "renewal" action. Includes an ID reference to a relevant onchain event, in case onchain event details are required.tempLogicalSubregistryActionwhich is an internal-only table which helps connecting all state transitions spread across multiple onchain events into a single "logical" registrar action record.ENS Referrals
The
@namehash/ens-referralspackage has become a dependency for:@ensnode/ensnode-sdkpackage (which also re-exports some types and functions for the benefit of clients, i.e.ensindexerapp).ENSNode SDK
The main update is the new
registrar-actionsmodule atpackages/ensnode-sdk/src/registrar-actions.ENSIndexer
The
registrarsplugin has been created and splits into two parts:apps/ensindexer/src/plugins/registrars/ethnamesapps/ensindexer/src/plugins/registrars/basenamesapps/ensindexer/src/plugins/registrars/lineanamesapps/ensindexer/src/plugins/registrars/sharedModules include:
handlerswith event handlers defined for indexed:libwith module-specific helpers