Skip to content

Deploy standalone mongodb exporter #931

Open
muzammil360 wants to merge 72 commits intofeature/storage-clusterfrom
muz/mongodb-exporter
Open

Deploy standalone mongodb exporter #931
muzammil360 wants to merge 72 commits intofeature/storage-clusterfrom
muz/mongodb-exporter

Conversation

@muzammil360
Copy link
Copy Markdown
Contributor

@muzammil360 muzammil360 commented Feb 11, 2026

This pull request introduces support for monitoring MongoDB resources in the infrastructure, automating the deployment of MongoDB exporters, services, and ServiceMonitors, and integrating a new Grafana dashboard for MongoDB. The changes ensure that MongoDB metrics are collected and visualized alongside existing MySQL monitoring, and update the configuration and templates accordingly.

MongoDB Monitoring Integration

  • Added a new Terraform local and resource to identify and process MongoDB resources for monitoring (monolith_mongodb_resources_to_monitor) and to generate corresponding monitoring manifests. [1] [2] [3]
  • Created a new template (monolith-mongodb-monitoring.yaml.tpl) that defines a Deployment for the MongoDB exporter, a Service for metrics exposure, and a ServiceMonitor for Prometheus scraping.
  • Updated the kustomization template to include the generated MongoDB monitoring YAML files, ensuring they are applied during deployment.

Grafana Dashboard Updates

  • Switched the MongoDB Grafana dashboard to use a new ID and revision, and updated the dashboard manifest to specify the required Prometheus datasource. [1] [2]
  • Re-enabled the MongoDB Grafana dashboard in the monitoring configuration, replacing previously commented-out, non-functional definitions with a working one.

@muzammil360 muzammil360 changed the base branch from main to feature/storage-cluster February 11, 2026 08:12
@muzammil360 muzammil360 marked this pull request as draft February 11, 2026 09:35
@muzammil360 muzammil360 force-pushed the muz/mongodb-exporter branch 2 times, most recently from fdeb1cc to 6b89ed6 Compare February 21, 2026 09:44
@muzammil360 muzammil360 requested a review from Copilot February 26, 2026 09:59
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds MongoDB DBaaS monitoring support in the monolith environment by generating Kubernetes manifests (exporter Deployment/Service/ServiceMonitor) via Terraform and enabling a MongoDB Grafana dashboard, mirroring the existing MySQL monitoring approach.

Changes:

  • Extend the stateful-resources Kustomize template to include generated MongoDB monitoring manifests.
  • Add Terraform generation of per-MongoDB monitoring manifests using a new monolith-mongodb-monitoring.yaml.tpl template.
  • Enable a MongoDB Grafana dashboard by switching to a new grafana.com dashboard ID/revision in both the template and rendered GitOps manifest.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
terraform/gitops/stateful-resources/templates/stateful-resources/stateful-resources-kustomization.yaml.tpl Includes generated MongoDB monitoring YAMLs in the kustomization resources list.
terraform/gitops/stateful-resources/templates/stateful-resources/monolith-mongodb-monitoring.yaml.tpl New template defining Deployment/Service/ServiceMonitor for mongodb_exporter.
terraform/gitops/stateful-resources/stateful-resources-config.tf Wires new locals and a local_file generator for MongoDB monitoring manifests.
terraform/gitops/generate-files/templates/monitoring/post-config/dashboards/default.yaml.tpl Enables a MongoDB GrafanaDashboard (grafana.com import).
gitops/applications/base/monitoring-post-config/dashboards-default.yaml Updates the rendered dashboard ID/revision for MongoDB.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +375 to +376
db_secret_key = "MONGODB_CLUSTER_ADMIN_PASSWORD"
db_username_key = "MONGODB_CLUSTER_ADMIN_USER"
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

db_secret_key/db_username_key are hardcoded to MONGODB_CLUSTER_ADMIN_PASSWORD/MONGODB_CLUSTER_ADMIN_USER, but the monolith DB definitions use external_resource_config.master_user_password_secret_key (e.g. MONGODB_USER_ADMIN_PASSWORD). This will make the generated exporter Deployment reference non-existent secret keys for DBaaS/monolith resources. Prefer wiring db_secret_key to each.value.external_resource_config.master_user_password_secret_key and take the username from each.value.external_resource_config.username (or ensure the referenced secret actually contains the admin user/password keys).

Suggested change
db_secret_key = "MONGODB_CLUSTER_ADMIN_PASSWORD"
db_username_key = "MONGODB_CLUSTER_ADMIN_USER"
db_secret_key = each.value.external_resource_config.master_user_password_secret_key

Copilot uses AI. Check for mistakes.
Comment thread terraform/gitops/stateful-resources/stateful-resources-config.tf
Comment on lines +20 to +38
- --mongodb.uri=mongodb://$(MONGODB_USERNAME):$(MONGODB_PASSWORD)@${externalservice_name}.${namespace}:${port}/admin?replicaSet=rs0&tlsInsecure=true&ssl=true
- --mongodb.direct-connect=true
- --compatible-mode
- --collect-all
- --log.level=info
ports:
- name: metrics
containerPort: 9216
env:
- name: MONGODB_USERNAME
valueFrom:
secretKeyRef:
name: ${db_secret}
key: ${db_username_key}
- name: MONGODB_PASSWORD
valueFrom:
secretKeyRef:
name: ${db_secret}
key: ${db_secret_key}
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The exporter URI builds the username from $(MONGODB_USERNAME) sourced from ${db_secret} using ${db_username_key}. For monolith DB configs, only the password key is currently provided (master_user_password_secret_key), so this is likely to fail due to a missing username key in the secret. Consider using the ${db_username} template var directly in the URI and only source the password from the secret (or ensure the secret contains the referenced username key).

Copilot uses AI. Check for mistakes.
Comment on lines +20 to +42
- --mongodb.uri=mongodb://$(MONGODB_USERNAME):$(MONGODB_PASSWORD)@${externalservice_name}.${namespace}:${port}/admin?replicaSet=rs0&tlsInsecure=true&ssl=true
- --mongodb.direct-connect=true
- --compatible-mode
- --collect-all
- --log.level=info
ports:
- name: metrics
containerPort: 9216
env:
- name: MONGODB_USERNAME
valueFrom:
secretKeyRef:
name: ${db_secret}
key: ${db_username_key}
- name: MONGODB_PASSWORD
valueFrom:
secretKeyRef:
name: ${db_secret}
key: ${db_secret_key}
volumeMounts:
- name: ca-bundle-volume
mountPath: /etc/mongodb-certs
readOnly: true
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A CA bundle secret is mounted at /etc/mongodb-certs, but the connection string enables tlsInsecure=true and does not reference the mounted CA file. This effectively disables TLS verification and makes the CA mount unused. Prefer configuring the MongoDB client to use the mounted CA (e.g., tls=true + CA file option) and drop tlsInsecure=true so traffic is verified.

Copilot uses AI. Check for mistakes.
@muzammil360 muzammil360 force-pushed the muz/mongodb-exporter branch from 989bb09 to ef47ad4 Compare March 31, 2026 05:42
@muzammil360 muzammil360 marked this pull request as ready for review April 7, 2026 09:55
Copy link
Copy Markdown
Collaborator

@KhaledSaiidi KhaledSaiidi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants