Monitor TapData with Prometheus
Use Prometheus to collect TapData metrics, Grafana to review trends, and Alertmanager to send notifications. You can also monitor the MongoDB system database used by TapData.
Choose a starting point
| Goal | Read this page |
|---|---|
| Deploy Prometheus, Grafana, and Alertmanager for the first time | Deploy Prometheus monitoring |
| Connect TapData to an existing Prometheus, Grafana, or Alertmanager deployment | Use an existing monitoring stack |
| Understand metric values, trends, and healthy ranges | Metric reference and health assessment |
| Import a TapData, API Server, or MongoDB dashboard | Use the Grafana dashboards |
| Monitor the MongoDB system database used by TapData | Configure MongoDB monitoring |
| Configure severity-based alerts, notifications, routine checks, and incident response | Configure alerts and routine monitoring checks |
If deployment is complete but a dashboard shows No data, go directly to Troubleshoot missing data.
Recommended workflow
For a first-time integration, complete these tasks in order:
- Enable and verify the metric endpoints. Add only endpoints that return Prometheus metrics.
- Deploy the monitoring services in this guide, or use an existing monitoring stack.
- Review the metrics available in your environment before enabling task, API Server, or MongoDB dashboards and alerts.
- Download and import the Grafana templates. Validate one instance against the TapData UI before selecting All.
- Install the alert rules, adjust thresholds to the business SLA, and test the notification path.
- Use the routine monitoring checklist for ongoing operations and incident escalation.
Common terms
| Term | Meaning in this guide |
|---|---|
| Target | A metric endpoint that Prometheus accesses on a schedule. |
| Metric endpoint | An HTTP URL that returns monitoring data, such as the Flow Engine /actuator/prometheus endpoint. |
| Scrape | The process in which Prometheus accesses a metric endpoint and stores the returned data. |
| Metric | Queryable runtime data, such as service state, CPU usage, or task lag. |
| Label | Additional information used to filter a metric, such as project, instance, or task_name. |
| PromQL | The Prometheus query language. Run blocks labeled promql on the Prometheus query page. |
| Exporter | A program that converts the state of a system such as MongoDB into Prometheus metrics. |
| No data | The query found no matching data. It does not mean that the metric value is 0. |
| SLA | A service objective defined by the business, such as the maximum permitted task lag. |
Monitoring coverage
| Object | Primary signals |
|---|---|
| Management | Availability, HTTP requests, JVM, process, and host resources |
| Flow Engine | Availability, JVM, process, and host resources |
| TapData tasks | Task status, connection status, CDC lag, node processing time, and startup milestones |
| Agent | Availability, process, and host resources |
| API Server | HTTP requests, Node.js runtime, process, and log metrics |
| MongoDB | Database connection, replica set, connections, operations, and WiredTiger cache |
Validate the monitoring setup
After deployment, confirm the following results:
- Under Status > Target health in Prometheus, every required target is UP. Remove scrape jobs for components that are not deployed.
- The Grafana data source test succeeds, and dashboard variables list the expected environment and instances.
- Dashboard values, units, state mappings, and task counts pass the environment comparison.
- Alerts use only verified metrics, and task-lag thresholds match the business SLA.
- During a notification test, alerts move from Pending to Firing, reach the expected receiver, and send a resolved notification after recovery.
- On-call engineers understand the response order, incident classification, and escalation conditions.