feat(tools/talis): vendor talis deployment tool + Fibre experiment runner (#3301)

Brings the celestia-app talis multi-cloud deploy tool into ev-node,
plus a long-lived ev-node aggregator runner that wires the existing
celestia-node-fiber adapter behind ev-node's DA client interface.
Verified end-to-end on AWS — talis up → genesis → deploy →
setup-fibre → start-fibre → fibre-bootstrap-evnode reaches
24.57 MB/s @ 99.7 % ok on a 60 s sustained loadgen
(3 × c6in.4xlarge validators + c6in.2xlarge bridge +
c6in.8xlarge ev-node + c6in.2xlarge load-gen, us-east-1).

What this adds:

  • tools/talis/                — vendored from celestia-app's
    feat/fibre-payments. Provisions AWS / DO / GCP boxes for
    validators + bridge + ev-node + load-gen, deploys binaries +
    init scripts, drives the Fibre setup-fibre + start-fibre flow,
    and ships a fibre-bootstrap-evnode step that scp's the bridge
    JWT and Fibre payment keyring onto each ev-node before its
    init script starts the daemon.
  • tools/celestia-node-fiber/cmd/evnode-fibre/  — the long-lived
    aggregator runner. Wires block.NewFiberDAClient on top of the
    celestia-node-fiber adapter that julien/fiber already ships,
    plus the in-memory executor + HTTP /tx ingress used by
    evnode-txsim. Distinct from the existing fiber-bench cmd.
  • tools/talis/cmd/evnode-txsim/ — small Go load-gen that pumps
    the runner's HTTP /tx ingress for a fixed duration; deployed
    to load-gen boxes and prints a single TXSIM: line on completion.

Two small ev-node-side helpers the runner calls:

  • block/public.go: SetMaxBlobSize(n) — overrides the per-blob
    byte cap so the runner can lift Celestia's 5 MiB default to
    Fibre's 120 MiB headroom.
  • pkg/config/config.go: Config.ApplyFiberDefaults() — flips the
    DA config to Fibre-friendly settings (adaptive batching, 1 s
    DA.BlockTime, 50-deep pending-cache window) when the Fiber
    profile is enabled, so a runner can opt in with one call.

setup-fibre robustness fixes uncovered during the verified run:

  • bash script for set-host now retries until the validator's
    host appears in `query valaddr providers`. The previous one-
    shot call relied on `--yes` returning the txhash before block
    inclusion; if the chain wasn't ready, the tx silently bounced.
    The Fibre client cached the partial set on startup and uploads
    cascaded to "host not found" → "voting power: collected 0".
  • talis-CLI side polls `query valaddr providers` after the per-
    validator scripts finish and refuses to return until all
    validators are registered (5-minute deadline).

External dependency (documented in tools/talis/fibre.md):

  • Sibling clone of celestia-app on a branch with feat/fibre-payments
    + sysrex/fibre_url_fix cherry-picked. Without the URL-parse fix
    the Fibre client rejects every host:port registration.

Tested:
  - go build ./... — clean
  - go test ./block/internal/submitting ./pkg/config (the two
    pre-existing test failures on julien/fiber — TestAddFlags
    and TestFiberClient_Submit_BlobTooLarge — are not introduced
    by this PR and reproduce on raw julien/fiber)
  - End-to-end AWS deploy from this branch — 24.57 MB/s, 99.7 % ok

Author: walldiss

block/public.go

@@ -19,6 +19,14 @@ func DefaultBlockOptions() BlockOptions {
 	return common.DefaultBlockOptions()
 }
 
+// SetMaxBlobSize overrides the per-blob byte cap used by the executor
+// and DA submitter when sizing batches and validating individual blobs.
+// Intended for one-shot startup wiring (e.g. to lift Celestia's 5 MiB
+// default to Fibre's 120 MiB headroom).
+func SetMaxBlobSize(n uint64) {
+	common.DefaultMaxBlobSize = n
+}
+
 // Expose Metrics for constructor
 type Metrics = common.Metrics
 

go.work

@@ -0,0 +1,6 @@
+go 1.25.7
+
+use (
+	.
+	./core
+)

go.work.sum

@@ -343,6 +343,33 @@ func (d *DAConfig) IsFiberEnabled() bool {
 	return d.Fiber.Enabled
 }
 
+// ApplyFiberDefaults flips the DA client to Fiber-friendly defaults
+// when the Fiber profile is enabled — adaptive batching, a 1 s
+// DA.BlockTime so inclusion-tracking keeps pace with Fibre's
+// settlement, and a bounded pending-cache window so a Fibre stall
+// can't grow memory unbounded. Caller-provided non-zero values for
+// the tunables (BatchSizeThreshold, BatchMinItems) are preserved.
+//
+// Intended to be invoked once at runner startup, after parsing the
+// usual config but before constructing the DA client.
+func (c *Config) ApplyFiberDefaults() {
+	if !c.DA.IsFiberEnabled() {
+		return
+	}
+
+	c.DA.BatchingStrategy = "adaptive"
+	if c.DA.BatchSizeThreshold <= 0 || c.DA.BatchSizeThreshold > 1 {
+		c.DA.BatchSizeThreshold = 0.5
+	}
+	c.DA.BatchMaxDelay = DurationWrapper{Duration: 8 * time.Second}
+	if c.DA.BatchMinItems == 0 {
+		c.DA.BatchMinItems = 1
+	}
+
+	c.DA.BlockTime = DurationWrapper{Duration: 1 * time.Second}
+	c.Node.MaxPendingHeadersAndData = 50
+}
+
 // GetNamespace returns the namespace for header submissions.
 func (d *DAConfig) GetNamespace() string {
 	return d.Namespace