k0rdent · gmlexx · Apr 13, 2026 · Apr 13, 2026 · Apr 13, 2026 · Apr 13, 2026
@@ -1,6 +1,6 @@
 ---
 name: kind-deploy
-description: Deploy KCM and KOF on local kind clusters for development and testing. Covers the full workflow from prerequisites through mothership setup, local registry, chart publishing, and optional adopted regional/child cluster deployment.
+description: Deploy KCM and KOF on local kind clusters for development and testing. Covers the full workflow from prerequisites through mothership setup, local registry, chart publishing, optional Istio service mesh, and optional adopted regional/child cluster deployment.
 license: Apache-2.0
 compatibility: opencode
 metadata:
@@ -15,20 +15,22 @@ Guide developers through a complete local KCM + KOF deployment using kind cluste
 - Prerequisites and sibling repo layout (`../kcm` next to `../kof`)
 - Optional squid proxy and Docker registry auth setup
 - KCM bootstrap on the mothership kind cluster (`kcm-dev`)
+- Optional Istio service mesh setup (`../istio`)
 - Local OCI registry deployment and Helm chart packaging/pushing
 - KOF mothership deployment via `dev-deploy`
-- Optional adopted regional and child kind cluster setup
-- CoreDNS patching for cross-cluster name resolution
+- Optional adopted regional and child kind cluster setup (with or without Istio)
+- CoreDNS patching for cross-cluster name resolution (non-Istio path)
 - Monitoring and verification commands
 
 ## When to use me
 
 Use this skill when a developer asks to:
 - Set up a local dev environment for KOF or KCM
 - Deploy KOF on kind clusters
+- Deploy KOF with Istio service mesh
 - Troubleshoot or re-run the local dev deployment workflow
-- Deploy adopted regional/child clusters locally
-- Understand what `make kcm-dev-apply`, `make dev-deploy`, etc. actually do
+- Deploy adopted regional/child clusters locally (with or without Istio)
+- Understand what `make kcm-dev-apply`, `make dev-deploy`, `make dev-istio-regional-deploy-adopted`, etc. actually do
 
 ---
 
@@ -39,7 +41,7 @@ When the user asks to deploy with **`auto`** (e.g. "deploy KOF auto", "run the f
 **Non-optional steps run in auto mode (in order):**
 
 1. `make kcm-dev-apply` — Deploy KCM on the mothership kind cluster
-2. `make helm-push` — Build and push Helm charts
+2. `make helm-push` — Build and push KOF Helm charts
 3. Add `/etc/hosts` entry for `dex.example.com`
 4. Start `cloud-provider-kind` container
 5. `make dev-deploy` — Deploy KOF mothership
@@ -49,15 +51,17 @@ When the user asks to deploy with **`auto`** (e.g. "deploy KOF auto", "run the f
 - Step 1: `make cli-install` (run manually if tools are missing)
 - Step 2: Docker pull-limit workaround
 - Step 3: Squid proxy setup
-- Step 9: Adopted regional cluster deployment
-- Step 10: Adopted child cluster deployment
+- Step 5: Istio service mesh setup
+- Step 10: Adopted regional cluster deployment
+- Step 11: Adopted child cluster deployment
 
 **Behavior in auto mode:**
 
 - Run each non-optional step using the Bash tool immediately, without prompting.
 - If a step fails, stop and report the error with the relevant logs/output.
 - After all steps complete, print a summary of what was done and verification commands.
-- If the user says `auto` together with optional steps (e.g. "auto with regional cluster"), include those optional steps in the sequence.
+- If the user says `auto` together with optional steps (e.g. "auto with Istio", "auto with regional cluster"), include those optional steps in the sequence.
+- If Istio is requested, run Step 5 after Step 4 and before Step 6. When deploying adopted clusters, use the Istio variants (`make dev-istio-regional-deploy-adopted`, `make dev-istio-child-deploy-adopted`).
 
 ---
 
@@ -71,7 +75,8 @@ When the user asks to deploy with **`auto`** (e.g. "deploy KOF auto", "run the f
    ```
    ~/work/
    ├── kcm/    # https://github.com/k0rdent/kcm
-   └── kof/    # this repo (or your fork)
+   ├── kof/    # this repo (or your fork)
+   └── istio/  # https://github.com/k0rdent/istio (optional, only for Istio setup)
    ```
    The Makefile variable `KCM_REPO_PATH` defaults to `../kcm`.
 
@@ -143,7 +148,72 @@ kubectl get mgmt kcm
 
 ---
 
-## Step 5 — Build and push Helm charts
+## Step 5 — Optional: Set up Istio service mesh
+
+Skip this step if you do not need Istio. When Istio is used, some later steps differ — see Steps 10 and 11.
+
+This step must be run **before** `make dev-deploy` (Step 9) because the `kof` namespace must be pre-created with the Istio injection label before KOF is installed. If Helm creates the namespace via `--create-namespace`, the injection label will not be set and sidecar injection will not work.
+
+### 5a — Pre-create the KOF namespace with injection enabled
+
+```bash
+kubectl create namespace kof
+kubectl label namespace kof istio-injection=enabled
+```
+
+### 5b — Clone the istio repo (if not already done)
+
+```bash
+cd ..
+git clone https://github.com/k0rdent/istio.git
+cd istio
+```
+
+### 5c — Build and push Istio charts to the local registry
+
+```bash
+make cli-install
+make helm-push
+```
+
+This publishes Istio Helm charts to the shared local OCI registry at `oci://127.0.0.1:5001/charts` (the same registry used by KOF).
+
+### 5d — Build the istio-operator Docker image
+
+```bash
+make istio-operator-docker-build
+```
+
+### 5e — Install the `k0rdent-istio` Helm chart
+
+```bash
+helm upgrade --create-namespace --install --wait k0rdent-istio ./charts/k0rdent-istio \
+  -n istio-system \
+  --set k0rdent-istio.repo.spec.url=oci://kcm-local-registry:5000/charts \
+  --set k0rdent-istio.repo.spec.type=oci \
+  --set k0rdent-istio.repo.spec.insecure=true \
+  --set operator.image.registry=docker.io/library \
+  --set operator.image.repository=istio-operator-controller \
+  --set "istiod.meshConfig.extensionProviders[0].name=otel-tracing" \
+  --set "istiod.meshConfig.extensionProviders[0].opentelemetry.port=4317" \
+  --set "istiod.meshConfig.extensionProviders[0].opentelemetry.service=kof-collectors-daemon-collector.kof.svc.cluster.local" \
+  --set-json 'gateway.resource.spec.servers[0]={"port":{"number":15443,"name":"tls","protocol":"TLS"},"tls":{"mode":"AUTO_PASSTHROUGH"},"hosts":["{clusterName}-vmauth.kof.svc.cluster.local"]}'
+```
+
+The OTel tracing extension provider points to the KOF collectors daemon so that Istio can export traces directly via the OpenTelemetry protocol.
+
+**To verify:**
+
+```bash
+kubectl get pod -n istio-system
+kubectl get helmrelease -n istio-system
+```
+
+---
+
+## Step 6 — Build and push KOF Helm charts
+
+From the `kof/` repo root:
 
 ```bash
 make helm-push
@@ -153,7 +223,7 @@ Runs `helm dependency update` + `helm lint` + `helm package` for every chart und
 
 ---
 
-## Step 6 — Update local DNS for Dex
+## Step 7 — Update local DNS for Dex
 
 ```bash
 grep -qxF "127.0.0.1 dex.example.com" /etc/hosts || \
@@ -162,7 +232,7 @@ grep -qxF "127.0.0.1 dex.example.com" /etc/hosts || \
 
 ---
 
-## Step 7 — Run cloud-provider-kind (external IP for gateways)
+## Step 8 — Run cloud-provider-kind (external IP for gateways)
 
 ```bash
 docker start cloud-provider-kind 2>/dev/null || \
@@ -175,7 +245,7 @@ This allocates external IPs for `LoadBalancer` services inside kind clusters. Th
 
 ---
 
-## Step 8 — Deploy KOF mothership
+## Step 9 — Deploy KOF mothership
 
 ```bash
 make dev-deploy
@@ -194,6 +264,8 @@ What this does:
 10. Waits for all HelmReleases in `kof` namespace to be `Ready` (10 min)
 11. Restarts `kof-mothership-kof-operator` and `kof-mothership-dex` deployments
 
+> **Note (Istio):** If you ran Step 5, the `kof` namespace already exists with the injection label. Helm's `--create-namespace` is a no-op in that case and the label is preserved.
+
 **Optional env vars for `dev-deploy`:**
 
 | Var | Effect |
@@ -213,15 +285,17 @@ kubectl get pod -n kof
 
 ---
 
-## Step 9 (Optional) — Deploy adopted regional cluster
+## Step 10 (Optional) — Deploy adopted regional cluster
 
 ```bash
 make dev-adopted-deploy KIND_CLUSTER_NAME=regional-adopted
 ```
 
 Creates a second kind cluster `regional-adopted` using `config/kind-adopted.yaml`, registers it as an adopted cluster in KCM by generating credentials from `demo/creds/adopted-credentials.yaml`, and loads the OTel collector image into it.
 
-Then deploy the regional ClusterDeployment:
+Then deploy the regional ClusterDeployment — choose the variant that matches your setup:
+
+### Without Istio
 
 ```bash
 make dev-regional-deploy-adopted
@@ -232,6 +306,14 @@ This:
 2. Applies the ClusterDeployment to KCM
 3. Waits for `cert-manager` and `envoy-gateway` to deploy, then `kof-operators`, `kof-storage`, `kof-collectors` to upgrade on the regional cluster
 
+### With Istio
+
+```bash
+make dev-istio-regional-deploy-adopted
+```
+
+Uses the Istio-aware ClusterDeployment template instead. The same wait sequence applies.
+
 **Verify regional cluster:**
 
 ```bash
@@ -241,7 +323,11 @@ helm --kube-context=kind-regional-adopted list -A
 
 ---
 
-## Step 10 (Optional) — Deploy adopted child cluster
+## Step 11 (Optional) — Deploy adopted child cluster
+
+Choose the variant that matches your setup:
+
+### Without Istio
 
 ```bash
 make dev-adopted-deploy KIND_CLUSTER_NAME=child-adopted
@@ -253,6 +339,15 @@ make dev-child-deploy-adopted
 
 `dev-child-deploy-adopted` applies `demo/cluster/adopted-cluster-child.yaml` and waits for `cert-manager`, `kof-operators`, `kof-collectors` on the child.
 
+### With Istio
+
+```bash
+make dev-adopted-deploy KIND_CLUSTER_NAME=child-adopted
+make dev-istio-child-deploy-adopted
+```
+
+> **Note:** `make dev-coredns` is **not** required when using Istio — the Istio variant handles cross-cluster connectivity without the CoreDNS hostname patching step.
+
 **Optional:** `KOF_TENANT_ID=<id>` adds a tenant label to the ClusterDeployment.
 
 **Verify child cluster:**
@@ -273,6 +368,12 @@ make dev-deploy HELM_CHART_NAME=kof-mothership
 # Port-forward promxy for metrics inspection
 make dev-promxy-port-forward   # → localhost:8082
 
+# Deploy adopted regional cluster with Istio
+make dev-istio-regional-deploy-adopted
+
+# Deploy adopted child cluster with Istio (no dev-coredns needed)
+make dev-istio-child-deploy-adopted
+
 # Remove an adopted cluster
 make dev-adopted-rm KIND_CLUSTER_NAME=regional-adopted
 make dev-adopted-rm KIND_CLUSTER_NAME=child-adopted
@@ -293,10 +394,11 @@ make kcm-dev-upgrade
 | `dev/values-local.yaml` | Generated/patched working values (git-ignored) |
 | `dev/docker/config.json` | Docker Hub credentials for kind node image pulls |
 | `dev/dex.env` | Optional Google OIDC connector credentials |
-| `demo/cluster/` | ClusterDeployment YAML templates |
+| `demo/cluster/` | ClusterDeployment YAML templates (including Istio variants) |
 | `demo/creds/` | Adopted cluster credential templates |
-| `scripts/patch-coredns.bash` | CoreDNS hostname patching script |
+| `scripts/patch-coredns.bash` | CoreDNS hostname patching script (non-Istio path) |
 | `scripts/generate-dex-secret.bash` | Dex TLS secret generation script |
+| `../istio/charts/k0rdent-istio/` | Istio operator Helm chart (separate repo) |
 
 ---
 
@@ -394,5 +496,9 @@ python3 .agents/skills/troubleshoot/scripts/step12_workloads.py support-bundle-<
 - Ensure `cloud-provider-kind` container is running
 - `kubectl --context=kind-regional-adopted get gateway -n kof`
 
+**Istio sidecar injection not working after `dev-deploy`**
+- The `kof` namespace must have the injection label set **before** `dev-deploy` runs
+- If the namespace was created by Helm (without the label), restart all pods in `kof` namespace to enable istio sidecars: `kubectl delete pod -n kof --all`
+
 **`dev-adopted-deploy` fails with `base64: invalid option -- w`** (macOS)
 - The Makefile uses `base64 -w 0` (Linux flag). On macOS, `base64` wraps by default but `-w` is not supported. Install GNU coreutils: `brew install coreutils` and ensure `gbase64` or `base64` from coreutils is on `$PATH`, or run on Linux.
@@ -247,7 +247,7 @@ dev-deploy: dev kof-namespace ## Deploy KOF umbrella chart with local developmen
 		$(MAKE) kof-operator-docker-build; \
 	fi
 	cp -f $(TEMPLATES_DIR)/$(KOF_VALUES) dev/values-local.yaml
-	@if $(KUBECTL) get namespace -l istio-injection=true | grep -q 'kof'; then \
+	@if $(KUBECTL) get namespace -l istio-injection=enabled | grep -q 'kof'; then \
 		echo "⚠️ Istio enabled, disable cert-manager installation"; \
 		$(YQ) eval -i '.kof-mothership.values.cert-manager-service-template.enabled = false' dev/values-local.yaml; \
 	fi