Integrate with Prometheus

On this page

Prerequisites

Procedure
Example Configurations
Import a Sample Grafana Dashboard
Performance Metrics Available to Prometheus
MongoDB Metric Labels
MongoDB Information Metrics
Hardware Metrics
Hardware Metric Labels

Prometheus collects metrics from configured targets at given intervals, evaluates rule expressions, displays the results, and can trigger alerts when it observes specific conditions.

Our integration allows you to configure Ops Manager to send metric data about your deployment to your Prometheus instance.

Prerequisites

Prometheus integration is available in automation managed clusters that use MongoDB Agent 12.0.15.7646 or later. MongoDB Agent 12.0.15.7646 is released with Ops Manager 6.0.7.
Have a working Prometheus instance. To set up a working instance, see their Installation Guide.
(Optional) Use Grafana to visualize your Prometheus metrics.

Procedure

To integrate Ops Manager with Prometheus:

Navigate to the Integrations page for your project.

If it is not already displayed, select the organization that contains your desired project from the Organizations menu in the navigation bar.
If it is not already displayed, select your desired project from the Projects menu in the navigation bar.
Next to the Projects menu, expand the Options menu, then click Integrations.

Click Configure for the Prometheus integration card.

Enter your preferred username and password.

Prometheus authentication credentials are specifically designed for use with the Prometheus integration in Ops Manager. Ops Manager uses these credentials to only access the Prometheus discovery endpoint and scrape Prometheus metrics from Ops Manager nodes. They are strictly limited to these functions and do not have any additional permissions or capabilities beyond accessing and collecting monitoring data.

Important

Copy your username and password in a secure location. You can't access the password after you leave this screen.

Enter your IP address and port.

Tip

The default value, 0.0.0.0:9216, scrapes metrics on port 9216 on all IPv4 addresses on the local machine.

(Optional) Encrypt all Prometheus metrics.

If you enable this setting, Ops Manager assures that your Prometheus instance uses https to scrape metrics.

Fields	Description
TLS Certificate Key File Path	`PEM` file path that contains certificate and key required to spin up a `https` Prometheus scraping endpoint. You are responsible for the following: TLS Certificate Key File issuance and renewal. Checking if the endpoint started correctly in the automation agent logs.
TLS Certificate Key File Password	Required if the certificate key file is encrypted.

Select your preferred service discovery method.

Discovery Method

Description

HTTP SD

This method requires Prometheus v2.28 and later. It generates the scrape_config part of your configuration file to discover targets over an HTTP endpoint.

File Service Discovery

This method allows Prometheus to read YAML or JSON documents to configure the targets to scrape from.

You are responsible for providing the targets by making a request to the Discovery API and storing its results in a targets.json file.

To make the request, substitute the placeholder text in one of the following tabs or create your own script in another language.

# Sets the `Authorization` header on every scrape
# request with the username and password from the
# previous step. The URL that Prometheus fetches the
# targets from.
# Replace the <group-id> with the project ID of your
# Atlas instance.
curl --header 'Accept: application/json' \
     --user <username>:<password> \
     --request GET "https://cloud.mongodb.com/prometheus/v1.0/groups/{GROUP-ID}/discovery"

If you need to install the requests library, see their Installation Guide.

import time, json, requests
# This script sets the `Authorization` header on every
# scrape request with the configured username and
# password. Then it tells Prometheus to fetch targets
# from the specified URL.
#
# Note: Replace the <username> and <password> with the
# values in the previous step, and <group-id> with the
# project ID of your Atlas instance.
basic_auth_user="<username>"
basic_auth_password="<password>"
discovery_api_url="https://cloud.mongodb.com/prometheus/v1.0/groups/{GROUP-ID}/discovery"
# The script updates your targets.json file every
# minute, if it successfully retrieves targets.
#
# Note: Replace the <path-to-targets.json> with the
# path to your targets.json file.
starttime = time.time()
while True:
  r = requests.get(discovery_api_url, auth=(basic_auth_user, basic_auth_password))
  if  r.status_code == 200:
    with open('<path-to-targets.json>', 'w') as f:
      json.dump(r.json(), f)
  time.sleep(60.0 - ((time.time() - starttime) % 60.0))

To learn more about the Discovery API, see Return the Latest Targets for Prometheus.

Click Save.

View Your Cluster Metrics on Prometheus.

Copy the generated snippet into the scrape_configs section of your configuration file and substitute the placeholder text.
For an example of the configuration file in either method, see Example Configurations.
Restart your Prometheus instance.
In your Prometheus instance, click Status in the top navigation bar, and click Targets to see the metrics of your deployment.

Example Configurations

The following shows examples of the configuration file when you use the HTTP Service Discovery or File Service Discovery method.

The configuration file in both methods contains the following fields:

Field	Description
`scrape_interval`	Time that indicates how frequently to scrape targets. This setting supports a minimum time of 10s.
`job_name`	Human-readable label assigned to scraped metrics.
`metrics_path`	HTTP resource path that indicates where to fetch metrics from targets.
`scheme`	Your Prometheus protocol scheme configured for requests, either `http` or `https`. If you configure `https`, you must specify `tlsPemPath`.
`basic_auth`	Authorization header to use on every scrape request.

HTTP Service Discovery

The HTTP Service Discovery method also contains the http_sd_configs field with the following sub-fields:

Field	Description
`url`	URL from which Prometheus fetches the targets.
`refresh_interval`	Time that indicates when to re-query the endpoint.
`basic_auth`	Credentials to use for authenticating to the API server.

global:
  scrape_interval: 15s
scrape_configs:
  - job_name: "CM-Testing-mongo-metrics"
    scrape_interval: 10s
    metrics_path: /metrics
    scheme : https
    basic_auth:
      username: prom_user_61e6e34e93eac1632d39f457
      password: V7hTyLfkjwiWQbv
    http_sd_configs:
      - url: https://cloud.mongodb.com/prometheus/v1.0/groups/61e6e34e93eac1632d39f457/discovery
        refresh_interval: 60s
        basic_auth:
          username: prom_user_61e6e34e93eac1632d39f457
          password: V7hTyLfkjwiWQbv

File Service Discovery

The File Service Discovery method also contains the file_sd_configs field with the following sub-field:

Field	Description
`files`	List that contains the files from which to extract the metrics scraping targets.

global:
  scrape_interval: 15s
scrape_configs:
  - job_name: "CM-Testing-mongo-metrics"
    scrape_interval: 10s
    metrics_path: /metrics
    scheme : https
    basic_auth:
      username: prom_user_61e6e34e93eac1632d39f457
      password: V7hTyLfkjwiWQbv
    file_sd_configs:
      - files:
        - /usr/local/etc/targets.json

Import a Sample Grafana Dashboard

To import a sample Grafana dashboard:

Navigate to the Import screen.

In the Grafana UI, click the button, and then click Import.

Click Upload JSON File.

Copy over the `mongo-metrics.json` or the `hardware-metrics.json` file to create a dashboard.

mongo-metrics.json

1 {
2  "__inputs": [
3    {
4      "name": "DS_THANOS",
5      "label": "thanos",
6      "description": "",
7      "type": "datasource",
8      "pluginId": "prometheus",
9      "pluginName": "Prometheus"
10    }
11  ],
12  "__elements": [],
13  "__requires": [
14    {
15      "type": "grafana",
16      "id": "grafana",
17      "name": "Grafana",
18      "version": "8.5.6"
19    },
20    {
21      "type": "panel",
22      "id": "graph",
23      "name": "Graph (old)",
24      "version": ""
25    },
26    {
27      "type": "datasource",
28      "id": "prometheus",
29      "name": "Prometheus",
30      "version": "1.0.0"
31    },
32    {
33      "type": "panel",
34      "id": "table",
35      "name": "Table",
36      "version": ""
37    }
38  ],
39  "annotations": {
40    "list": [
41      {
42        "builtIn": 1,
43        "datasource": {
44          "type": "datasource",
45          "uid": "grafana"
46        },
47        "enable": true,
48        "hide": true,
49        "iconColor": "rgba(0, 211, 255, 1)",
50        "name": "Annotations & Alerts",
51        "target": {
52          "limit": 100,
53          "matchAny": false,
54          "tags": [],
55          "type": "dashboard"
56        },
57        "type": "dashboard"
58      }
59    ]
60  },
61  "editable": true,
62  "fiscalYearStartMonth": 0,
63  "graphTooltip": 0,
64  "id": null,
65  "iteration": 1657202032408,
66  "links": [],
67  "liveNow": false,
68  "panels": [
69    {
70      "collapsed": false,
71      "datasource": {
72        "type": "prometheus",
73        "uid": "000000010"
74      },
75      "gridPos": {
76        "h": 1,
77        "w": 24,
78        "x": 0,
79        "y": 0
80      },
81      "id": 10,
82      "panels": [],
83      "title": "Overview",
84      "type": "row"
85    },
86    {
87      "datasource": {
88        "type": "prometheus",
89        "uid": "${DS_THANOS}"
90      },
91      "fieldConfig": {
92        "defaults": {
93          "color": {
94            "mode": "thresholds"
95          },
96          "custom": {
97            "align": "auto",
98            "displayMode": "auto",
99            "inspect": false
100          },
101          "decimals": 2,
102          "displayName": "",
103          "mappings": [],
104          "thresholds": {
105            "mode": "absolute",
106            "steps": [
107              {
108                "color": "green",
109                "value": null
110              },
111              {
112                "color": "red",
113                "value": 80
114              }
115            ]
116          },
117          "unit": "short"
118        },
119        "overrides": [
120          {
121            "matcher": {
122              "id": "byName",
123              "options": "Value"
124            },
125            "properties": [
126              {
127                "id": "unit",
128                "value": "short"
129              },
130              {
131                "id": "decimals",
132                "value": 2
133              },
134              {
135                "id": "custom.align"
136              }
137            ]
138          }
139        ]
140      },
141      "gridPos": {
142        "h": 4,
143        "w": 24,
144        "x": 0,
145        "y": 1
146      },
147      "id": 41,
148      "options": {
149        "footer": {
150          "fields": "",
151          "reducer": [
152            "sum"
153          ],
154          "show": false
155        },
156        "showHeader": true
157      },
158      "pluginVersion": "8.5.6",
159      "targets": [
160        {
161          "datasource": {
162            "type": "prometheus",
163            "uid": "${DS_THANOS}"
164          },
165          "exemplar": false,
166          "expr": "label_replace( sum(mongodb_info{group_id=~\"$group_id\", cl_name=~\"$cl_name\", instance=~\"$host.*\"}) by (group_id, org_id, rs_nm, cl_name), \"hostname\", \"$1\", \"instance\", \"(.*)\")\n",
167          "format": "table",
168          "instant": true,
169          "interval": "",
170          "legendFormat": "",
171          "refId": "A"
172        }
173      ],
174      "title": "Group Metadata",
175      "transformations": [
176        {
177          "id": "organize",
178          "options": {
179            "excludeByName": {
180              "Time": true,
181              "Value #A": true,
182              "instance": true,
183              "rs_nm": false
184            },
185            "indexByName": {
186              "": 2,
187              "Time": 0,
188              "Value #A": 6,
189              "cl_name": 4,
190              "group_id": 3,
191              "org_id": 1,
192              "rs_nm": 5
193            },
194            "renameByName": {
195              "": "Group Name ",
196              "Time": "",
197              "cl_name": "Cluster Name",
198              "group_id": "Group Id",
199              "hostname": "Host",
200              "instance": "",
201              "org_id": "Org Id",
202              "process_port": "Port",
203              "replica_state": "ReplicaSet State",
204              "rs_nm": "ReplicaSet Name"
205            }
206          }
207        },
208        {
209          "id": "groupBy",
210          "options": {
211            "fields": {
212              "Cluster Name": {
213                "aggregations": [],
214                "operation": "groupby"
215              },
216              "Group Id": {
217                "aggregations": [],
218                "operation": "groupby"
219              },
220              "Group Name ": {
221                "aggregations": [],
222                "operation": "groupby"
223              },
224              "Host": {
225                "aggregations": [],
226                "operation": "groupby"
227              },
228              "Host ": {
229                "aggregations": [],
230                "operation": "groupby"
231              },
232              "Org Id": {
233                "aggregations": [],
234                "operation": "groupby"
235              },
236              "Port": {
237                "aggregations": [],
238                "operation": "groupby"
239              },
240              "Replica set state": {
241                "aggregations": [],
242                "operation": "groupby"
243              },
244              "ReplicaSet Name": {
245                "aggregations": [],
246                "operation": "groupby"
247              },
248              "ReplicaSet State": {
249                "aggregations": [],
250                "operation": "groupby"
251              },
252              "host ": {
253                "aggregations": [],
254                "operation": "groupby"
255              },
256              "hostname": {
257                "aggregations": [],
258                "operation": "groupby"
259              },
260              "instance": {
261                "aggregations": [],
262                "operation": "groupby"
263              },
264              "port": {
265                "aggregations": [],
266                "operation": "groupby"
267              },
268              "process_port": {
269                "aggregations": [],
270                "operation": "groupby"
271              },
272              "replica set": {
273                "aggregations": [],
274                "operation": "groupby"
275              },
276              "replica set state": {
277                "aggregations": [],
278                "operation": "groupby"
279              },
280              "replica_state": {
281                "aggregations": [],
282                "operation": "groupby"
283              },
284              "rs_nm": {
285                "aggregations": [],
286                "operation": "groupby"
287              }
288            }
289          }
290        },
291        {
292          "id": "merge",
293          "options": {
294            "reducers": []
295          }
296        }
297      ],
298      "type": "table"
299    },
300    {
301      "datasource": {
302        "type": "prometheus",
303        "uid": "${DS_THANOS}"
304      },
305      "fieldConfig": {
306        "defaults": {
307          "color": {
308            "mode": "thresholds"
309          },
310          "custom": {
311            "align": "auto",
312            "displayMode": "auto",
313            "inspect": false
314          },
315          "decimals": 0,
316          "displayName": "",
317          "mappings": [],
318          "thresholds": {
319            "mode": "absolute",
320            "steps": [
321              {
322                "color": "green",
323                "value": null
324              },
325              {
326                "color": "red",
327                "value": 80
328              }
329            ]
330          },
331          "unit": "none"
332        },
333        "overrides": []
334      },
335      "gridPos": {
336        "h": 6,
337        "w": 24,
338        "x": 0,
339        "y": 5
340      },
341      "id": 42,
342      "options": {
343        "footer": {
344          "fields": "",
345          "reducer": [
346            "sum"
347          ],
348          "show": false
349        },
350        "showHeader": true
351      },
352      "pluginVersion": "8.5.6",
353      "targets": [
354        {
355          "datasource": {
356            "type": "prometheus",
357            "uid": "${DS_THANOS}"
358          },
359          "exemplar": false,
360          "expr": "label_replace( sum(mongodb_info{group_id=~\"$group_id\", cl_name=~\"$cl_name\", instance=~\"$host.*\"}) by (instance, replica_state_name, process_port, rs_nm, process_type), \"hostname\", \"$1\", \"instance\", \"(.*):.*\")\n",
361          "format": "table",
362          "instant": true,
363          "interval": "",
364          "legendFormat": "",
365          "refId": "A"
366        }
367      ],
368      "title": "Cluster host list",
369      "transformations": [
370        {
371          "id": "organize",
372          "options": {
373            "excludeByName": {
374              "Time": true,
375              "Value #A": true,
376              "instance": false,
377              "rs_nm": false
378            },
379            "indexByName": {
380              "Time": 0,
381              "Value": 6,
382              "instance": 1,
383              "process_port": 2,
384              "process_type": 4,
385              "replica_state_name": 5,
386              "rs_nm": 3
387            },
388            "renameByName": {
389              "Time": "",
390              "Value": "",
391              "hostname": "Host",
392              "instance": "Hostname",
393              "process_port": "Port",
394              "process_type": "Process Type",
395              "replica_state": "ReplicaSet State",
396              "replica_state_name": "Replica State",
397              "rs_nm": "ReplicaSet Name"
398            }
399          }
400        },
401        {
402          "id": "groupBy",
403          "options": {
404            "fields": {
405              "Host": {
406                "aggregations": [],
407                "operation": "groupby"
408              },
409              "Host ": {
410                "aggregations": [],
411                "operation": "groupby"
412              },
413              "Hostname": {
414                "aggregations": [],
415                "operation": "groupby"
416              },
417              "Port": {
418                "aggregations": [],
419                "operation": "groupby"
420              },
421              "Process Type": {
422                "aggregations": [],
423                "operation": "groupby"
424              },
425              "Replica State": {
426                "aggregations": [],
427                "operation": "groupby"
428              },
429              "Replica set state": {
430                "aggregations": [],
431                "operation": "groupby"
432              },
433              "ReplicaSet Name": {
434                "aggregations": [],
435                "operation": "groupby"
436              },
437              "ReplicaSet State": {
438                "aggregations": [],
439                "operation": "groupby"
440              },
441              "host ": {
442                "aggregations": [],
443                "operation": "groupby"
444              },
445              "hostname": {
446                "aggregations": [],
447                "operation": "groupby"
448              },
449              "instance": {
450                "aggregations": [],
451                "operation": "groupby"
452              },
453              "port": {
454                "aggregations": [],
455                "operation": "groupby"
456              },
457              "process_port": {
458                "aggregations": [],
459                "operation": "groupby"
460              },
461              "replica set": {
462                "aggregations": [],
463                "operation": "groupby"
464              },
465              "replica set state": {
466                "aggregations": [],
467                "operation": "groupby"
468              },
469              "replica_state": {
470                "aggregations": [],
471                "operation": "groupby"
472              },
473              "rs_nm": {
474                "aggregations": [],
475                "operation": "groupby"
476              }
477            }
478          }
479        },
480        {
481          "id": "merge",
482          "options": {
483            "reducers": []
484          }
485        }
486      ],
487      "type": "table"
488    },
489    {
490      "collapsed": false,
491      "datasource": {
492        "type": "prometheus",
493        "uid": "000000010"
494      },
495      "gridPos": {
496        "h": 1,
497        "w": 24,
498        "x": 0,
499        "y": 11
500      },
501      "id": 8,
502      "panels": [],
503      "title": "Memory",
504      "type": "row"
505    },
506    {
507      "aliasColors": {},
508      "bars": false,
509      "dashLength": 10,
510      "dashes": false,
511      "datasource": {
512        "type": "prometheus",
513        "uid": "${DS_THANOS}"
514      },
515      "description": "The number of megabytes resident. MMAPv1: It is typical over time, on a dedicated database server, for this number to approach the amount of physical ram on the box. WiredTiger: In a standard deployment resident is the amount of memory used by the WiredTiger cache plus the memory dedicated to other in memory structures used by the mongod process. By default, mongod with WiredTiger reserves 50% of the total physical memory on the server for the cache and at steady state, WiredTiger tries to limit cache usage to 80% of that total. For example, if a server has 16GB of memory, WiredTiger will assume it can use 8GB for cache and at steady state should use about 6.5GB.",
516      "fieldConfig": {
517        "defaults": {
518          "links": [],
519          "unit": "decmbytes"
520        },
521        "overrides": []
522      },
523      "fill": 1,
524      "fillGradient": 0,
525      "gridPos": {
526        "h": 8,
527        "w": 12,
528        "x": 0,
529        "y": 12
530      },
531      "hiddenSeries": false,
532      "id": 2,
533      "legend": {
534        "avg": false,
535        "current": false,
536        "max": false,
537        "min": false,
538        "show": true,
539        "total": false,
540        "values": false
541      },
542      "lines": true,
543      "linewidth": 1,
544      "nullPointMode": "null",
545      "options": {
546        "alertThreshold": true,
547        "legend": {
548          "calcs": [],
549          "displayMode": "list",
550          "placement": "bottom"
551        },
552        "tooltip": {
553          "mode": "single"
554        }
555      },
556      "percentage": false,
557      "pluginVersion": "8.5.6",
558      "pointradius": 2,
559      "points": false,
560      "renderer": "flot",
561      "seriesOverrides": [],
562      "spaceLength": 10,
563      "stack": false,
564      "steppedLine": false,
565      "targets": [
566        {
567          "exemplar": true,
568          "expr": "label_replace(sum(mongodb_mem_resident{group_id=~\"$group_id\", cl_name=~\"$cl_name\", rs_nm=~\"$rs_nm\", instance=~\"$host.*\",  process_port=~\"$process_port\"}) by (instance, process_port) , \"hostname\", \"$1\", \"instance\", \"(.*)\")",
569          "interval": "",
570          "legendFormat": "{{hostname}}:{{process_port}}",
571          "refId": "A",
572          "datasource": {
573            "type": "prometheus",
574            "uid": "${DS_THANOS}"
575          }
576        }
577      ],
578      "thresholds": [],
579      "timeRegions": [],
580      "title": "Memory - Resident",
581      "tooltip": {
582        "shared": true,
583        "sort": 0,
584        "value_type": "individual"
585      },
586      "type": "graph",
587      "xaxis": {
588        "mode": "time",
589        "show": true,
590        "values": []
591      },
592      "yaxes": [
593        {
594          "format": "decmbytes",
595          "logBase": 1,
596          "show": true
597        },
598        {
599          "format": "short",
600          "logBase": 1,
601          "show": true
602        }
603      ],
604      "yaxis": {
605        "align": false
606      }
607    },
608    {
609      "aliasColors": {},
610      "bars": false,
611      "dashLength": 10,
612      "dashes": false,
613      "datasource": {
614        "type": "prometheus",
615        "uid": "${DS_THANOS}"
616      },
617      "description": "The virtual megabytes for the mongod process. MMAPv1: Generally virtual should be a little larger than mapped (or 2x with --journal), but if virtual is many gigabytes larger, it indicates that excessive memory is being used by other aspects than the memory mapping of files -- that would be bad/suboptimal. The most common case of usage of a high amount of memory for non-mapped is that there are very many connections to the database. Each connection has a thread stack and the memory for those stacks can add up to a considerable amount. WiredTiger: Generally virtual should be a little larger than mapped, but if virtual is many gigabytes larger, it indicates that excessive memory is being used by other aspects than the memory mapping of files -- that would be bad/suboptimal. The most common case of usage of a high amount of memory for non-mapped is that there are very many connections to the database. Each connection has a thread stack and the memory for those stacks can add up to a considerable amount.",
618      "fieldConfig": {
619        "defaults": {
620          "links": [],
621          "unit": "decmbytes"
622        },
623        "overrides": []
624      },
625      "fill": 1,
626      "fillGradient": 0,
627      "gridPos": {
628        "h": 8,
629        "w": 12,
630        "x": 12,
631        "y": 12
632      },
633      "hiddenSeries": false,
634      "id": 72,
635      "legend": {
636        "avg": false,
637        "current": false,
638        "max": false,
639        "min": false,
640        "show": true,
641        "total": false,
642        "values": false
643      },
644      "lines": true,
645      "linewidth": 1,
646      "nullPointMode": "null",
647      "options": {
648        "alertThreshold": true,
649        "legend": {
650          "calcs": [],
651          "displayMode": "list",
652          "placement": "bottom"
653        },
654        "tooltip": {
655          "mode": "single"
656        }
657      },
658      "percentage": false,
659      "pluginVersion": "8.5.6",
660      "pointradius": 2,
661      "points": false,
662      "renderer": "flot",
663      "seriesOverrides": [],
664      "spaceLength": 10,
665      "stack": false,
666      "steppedLine": false,
667      "targets": [
668        {
669          "exemplar": true,
670          "expr": "label_replace(sum(mongodb_mem_virtual{group_id=~\"$group_id\", cl_name=~\"$cl_name\", rs_nm=~\"$rs_nm\", instance=~\"$host.*\",  process_port=~\"$process_port\"}) by (instance, process_port) , \"hostname\", \"$1\", \"instance\", \"(.*)\")",
671          "interval": "",
672          "legendFormat": "{{hostname}}:{{process_port}}",
673          "refId": "A",
674          "datasource": {
675            "type": "prometheus",
676            "uid": "${DS_THANOS}"
677          }
678        }
679      ],
680      "thresholds": [],
681      "timeRegions": [],
682      "title": "Memory - Virtual",
683      "tooltip": {
684        "shared": true,
685        "sort": 0,
686        "value_type": "individual"
687      },
688      "type": "graph",
689      "xaxis": {
690        "mode": "time",
691        "show": true,
692        "values": []
693      },
694      "yaxes": [
695        {
696          "format": "decmbytes",
697          "logBase": 1,
698          "show": true
699        },
700        {
701          "format": "short",
702          "logBase": 1,
703          "show": true
704        }
705      ],
706      "yaxis": {
707        "align": false
708      }
709    },
710    {
711      "collapsed": false,
712      "datasource": {
713        "type": "prometheus",
714        "uid": "000000010"
715      },
716      "gridPos": {
717        "h": 1,
718        "w": 24,
719        "x": 0,
720        "y": 20
721      },
722      "id": 17,
723      "panels": [],
724      "title": "Asserts",
725      "type": "row"
726    },
727    {
728      "aliasColors": {},
729      "bars": false,
730      "dashLength": 10,
731      "dashes": false,
732      "datasource": {
733        "type": "prometheus",
734        "uid": "${DS_THANOS}"
735      },
736      "description": "The average rate of regular asserts raised per second over the selected sample period\n",
737      "fieldConfig": {
738        "defaults": {
739          "links": [],
740          "unit": "/s"
741        },
742        "overrides": []
743      },
744      "fill": 1,
745      "fillGradient": 0,
746      "gridPos": {
747        "h": 8,
748        "w": 6,
749        "x": 0,
750        "y": 21
751      },
752      "hiddenSeries": false,
753      "id": 73,
754      "legend": {
755        "avg": false,
756        "current": false,
757        "max": false,
758        "min": false,
759        "show": true,
760        "total": false,
761        "values": false
762      },
763      "lines": true,
764      "linewidth": 1,
765      "nullPointMode": "null",
766      "options": {
767        "alertThreshold": true,
768        "legend": {
769          "calcs": [],
770          "displayMode": "list",
771          "placement": "bottom"
772        },
773        "tooltip": {
774          "mode": "single"
775        }
776      },
777      "percentage": false,
778      "pluginVersion": "8.5.6",
779      "pointradius": 2,
780      "points": false,
781      "renderer": "flot",
782      "seriesOverrides": [],
783      "spaceLength": 10,
784      "stack": false,
785      "steppedLine": false,
786      "targets": [
787        {
788          "exemplar": true,
789          "expr": "label_replace(sum(rate(mongodb_asserts_regular{group_id=~\"$group_id\", cl_name=~\"$cl_name\", rs_nm=~\"$rs_nm\", instance=~\"$host.*\",  process_port=~\"$process_port\"}[$Interval])) by (instance, process_port) , \"hostname\", \"$1\", \"instance\", \"(.*)\")",
790          "interval": "",
791          "legendFormat": "{{hostname}}:{{process_port}}",
792          "refId": "A",
793          "datasource": {
794            "type": "prometheus",
795            "uid": "${DS_THANOS}"
796          }
797        }
798      ],
799      "thresholds": [],
800      "timeRegions": [],
801      "title": "Asserts - Regular",
802      "tooltip": {
803        "shared": true,
804        "sort": 0,
805        "value_type": "individual"
806      },
807      "type": "graph",
808      "xaxis": {
809        "mode": "time",
810        "show": true,
811        "values": []
812      },
813      "yaxes": [
814        {
815          "format": "/s",
816          "logBase": 1,
817          "show": true
818        },
819        {
820          "format": "short",
821          "logBase": 1,
822          "show": true
823        }
824      ],
825      "yaxis": {
826        "align": false
827      }
828    },
829    {
830      "aliasColors": {},
831      "bars": false,
832      "dashLength": 10,
833      "dashes": false,
834      "datasource": {
835        "type": "prometheus",
836        "uid": "${DS_THANOS}"
837      },
838      "description": "The average rate of warnings per second over the selected sample period",
839      "fieldConfig": {
840        "defaults": {
841          "links": [],
842          "unit": "/s"
843        },
844        "overrides": []
845      },
846      "fill": 1,
847      "fillGradient": 0,
848      "gridPos": {
849        "h": 8,
850        "w": 6,
851        "x": 6,
852        "y": 21
853      },
854      "hiddenSeries": false,
855      "id": 74,
856      "legend": {
857        "avg": false,
858        "current": false,
859        "max": false,
860        "min": false,
861        "show": true,
862        "total": false,
863        "values": false
864      },
865      "lines": true,
866      "linewidth": 1,
867      "nullPointMode": "null",
868      "options": {
869        "alertThreshold": true,
870        "legend": {
871          "calcs": [],
872          "displayMode": "list",
873          "placement": "bottom"
874        },
875        "tooltip": {
876          "mode": "single"
877        }
878      },
879      "percentage": false,
880      "pluginVersion": "8.5.6",
881      "pointradius": 2,
882      "points": false,
883      "renderer": "flot",
884      "seriesOverrides": [],
885      "spaceLength": 10,
886      "stack": false,
887      "steppedLine": false,
888      "targets": [
889        {
890          "exemplar": true,
891          "expr": "label_replace(sum(rate(mongodb_asserts_warning{group_id=~\"$group_id\", cl_name=~\"$cl_name\", rs_nm=~\"$rs_nm\", instance=~\"$host.*\",  process_port=~\"$process_port\"}[$Interval])) by (instance, process_port) , \"hostname\", \"$1\", \"instance\", \"(.*)\")",
892          "interval": "",
893          "legendFormat": "{{hostname}}:{{process_port}}",
894          "refId": "A",
895          "datasource": {
896            "type": "prometheus",
897            "uid": "${DS_THANOS}"
898          }
899        }
900      ],
901      "thresholds": [],
902      "timeRegions": [],
903      "title": "Asserts - Warning",
904      "tooltip": {
905        "shared": true,
906        "sort": 0,
907        "value_type": "individual"
908      },
909      "type": "graph",
910      "xaxis": {
911        "mode": "time",
912        "show": true,
913        "values": []
914      },
915      "yaxes": [
916        {
917          "format": "/s",
918          "logBase": 1,
919          "show": true
920        },
921        {
922          "format": "short",
923          "logBase": 1,
924          "show": true
925        }
926      ],
927      "yaxis": {
928        "align": false
929      }
930    },
931    {
932      "aliasColors": {},
933      "bars": false,
934      "dashLength": 10,
935      "dashes": false,
936      "datasource": {
937        "type": "prometheus",
938        "uid": "${DS_THANOS}"
939      },
940      "description": "The average rate of message asserts per second over the selected sample period. These are internal server errors that have a well defined text string. Stack traces are logged for these",
941      "fieldConfig": {
942        "defaults": {
943          "links": [],
944          "unit": "/s"
945        },
946        "overrides": []
947      },
948      "fill": 1,
949      "fillGradient": 0,
950      "gridPos": {
951        "h": 8,
952        "w": 6,
953        "x": 12,
954        "y": 21
955      },
956      "hiddenSeries": false,
957      "id": 75,
958      "legend": {
959        "avg": false,
960        "current": false,
961        "max": false,
962        "min": false,
963        "show": true,
964        "total": false,
965        "values": false
966      },
967      "lines": true,
968      "linewidth": 1,
969      "nullPointMode": "null",
970      "options": {
971        "alertThreshold": true,
972        "legend": {
973          "calcs": [],
974          "displayMode": "list",
975          "placement": "bottom"
976        },
977        "tooltip": {
978          "mode": "single"
979        }
980      },
981      "percentage": false,
982      "pluginVersion": "8.5.6",
983      "pointradius": 2,
984      "points": false,
985      "renderer": "flot",
986      "seriesOverrides": [],
987      "spaceLength": 10,
988      "stack": false,
989      "steppedLine": false,
990      "targets": [
991        {
992          "exemplar": true,
993          "expr": "label_replace(sum(rate(mongodb_asserts_msg{group_id=~\"$group_id\", cl_name=~\"$cl_name\", rs_nm=~\"$rs_nm\", instance=~\"$host.*\",  process_port=~\"$process_port\"}[$Interval])) by (instance, process_port) , \"hostname\", \"$1\", \"instance\", \"(.*)\")",
994          "interval": "",
995          "legendFormat": "{{hostname}}:{{process_port}}",
996          "refId": "A",
997          "datasource": {
998            "type": "prometheus",
999            "uid": "${DS_THANOS}"
1000          }
1001        }
1002      ],
1003      "thresholds": [],
1004      "timeRegions": [],
1005      "title": "Asserts - Message",
1006      "tooltip": {
1007        "shared": true,
1008        "sort": 0,
1009        "value_type": "individual"
1010      },
1011      "type": "graph",
1012      "xaxis": {
1013        "mode": "time",
1014        "show": true,
1015        "values": []
1016      },
1017      "yaxes": [
1018        {
1019          "format": "/s",
1020          "logBase": 1,
1021          "show": true
1022        },
1023        {
1024          "format": "short",
1025          "logBase": 1,
1026          "show": true
1027        }
1028      ],
1029      "yaxis": {
1030        "align": false
1031      }
1032    },
1033    {
1034      "aliasColors": {},
1035      "bars": false,
1036      "dashLength": 10,
1037      "dashes": false,
1038      "datasource": {
1039        "type": "prometheus",
1040        "uid": "${DS_THANOS}"
1041      },
1042      "description": "The average rate of user asserts per second over the selected sample period. These are errors that can be generated by a user such as out of disk space or duplicate key",
1043      "fieldConfig": {
1044        "defaults": {
1045          "links": [],
1046          "unit": "/s"
1047        },
1048        "overrides": []
1049      },
1050      "fill": 1,
1051      "fillGradient": 0,
1052      "gridPos": {
1053        "h": 8,
1054        "w": 6,
1055        "x": 18,
1056        "y": 21
1057      },
1058      "hiddenSeries": false,
1059      "id": 76,
1060      "legend": {
1061        "avg": false,
1062        "current": false,
1063        "max": false,
1064        "min": false,
1065        "show": true,
1066        "total": false,
1067        "values": false
1068      },
1069      "lines": true,
1070      "linewidth": 1,
1071      "nullPointMode": "null",
1072      "options": {
1073        "alertThreshold": true,
1074        "legend": {
1075          "calcs": [],
1076          "displayMode": "list",
1077          "placement": "bottom"
1078        },
1079        "tooltip": {
1080          "mode": "single"
1081        }
1082      },
1083      "percentage": false,
1084      "pluginVersion": "8.5.6",
1085      "pointradius": 2,
1086      "points": false,
1087      "renderer": "flot",
1088      "seriesOverrides": [],
1089      "spaceLength": 10,
1090      "stack": false,
1091      "steppedLine": false,
1092      "targets": [
1093        {
1094          "exemplar": true,
1095          "expr": "label_replace(sum(rate(mongodb_asserts_user{group_id=~\"$group_id\", cl_name=~\"$cl_name\", rs_nm=~\"$rs_nm\", instance=~\"$host.*\",  process_port=~\"$process_port\"}[$Interval])) by (instance, process_port) , \"hostname\", \"$1\", \"instance\", \"(.*)\")",
1096          "interval": "",
1097          "legendFormat": "{{hostname}}:{{process_port}}",
1098          "refId": "A",
1099          "datasource": {
1100            "type": "prometheus",
1101            "uid": "${DS_THANOS}"
1102          }
1103        }
1104      ],
1105      "thresholds": [],
1106      "timeRegions": [],
1107      "title": "Asserts - User",
1108      "tooltip": {
1109        "shared": true,
1110        "sort": 0,
1111        "value_type": "individual"
1112      },
1113      "type": "graph",
1114      "xaxis": {
1115        "mode": "time",
1116        "show": true,
1117        "values": []
1118      },
1119      "yaxes": [
1120        {
1121          "format": "/s",
1122          "logBase": 1,
1123          "show": true
1124        },
1125        {
1126          "format": "short",
1127          "logBase": 1,
1128          "show": true
1129        }
1130      ],
1131      "yaxis": {
1132        "align": false
1133      }
1134    },
1135    {
1136      "collapsed": false,
1137      "datasource": {
1138        "type": "prometheus",
1139        "uid": "000000010"
1140      },
1141      "gridPos": {
1142        "h": 1,
1143        "w": 24,
1144        "x": 0,
1145        "y": 29
1146      },
1147      "id": 44,
1148      "panels": [],
1149      "title": "Cache",
1150      "type": "row"
1151    },
1152    {
1153      "aliasColors": {},
1154      "bars": false,
1155      "dashLength": 10,
1156      "dashes": false,
1157      "datasource": {
1158        "type": "prometheus",
1159        "uid": "${DS_THANOS}"
1160      },
1161      "description": "The average rate of bytes per second read into WiredTiger's cache over the selected sample period.",
1162      "fieldConfig": {
1163        "defaults": {
1164          "links": [],
1165          "unit": "Bps"
1166        },
1167        "overrides": []
1168      },
1169      "fill": 1,
1170      "fillGradient": 0,
1171      "gridPos": {
1172        "h": 8,
1173        "w": 6,
1174        "x": 0,
1175        "y": 30
1176      },
1177      "hiddenSeries": false,
1178      "id": 77,
1179      "legend": {
1180        "avg": false,
1181        "current": false,
1182        "max": false,
1183        "min": false,
1184        "show": true,
1185        "total": false,
1186        "values": false
1187      },
1188      "lines": true,
1189      "linewidth": 1,
1190      "nullPointMode": "null",
1191      "options": {
1192        "alertThreshold": true,
1193        "legend": {
1194          "calcs": [],
1195          "displayMode": "list",
1196          "placement": "bottom"
1197        },
1198        "tooltip": {
1199          "mode": "single"
1200        }
1201      },
1202      "percentage": false,
1203      "pluginVersion": "8.5.6",
1204      "pointradius": 2,
1205      "points": false,
1206      "renderer": "flot",
1207      "seriesOverrides": [],
1208      "spaceLength": 10,
1209      "stack": false,
1210      "steppedLine": false,
1211      "targets": [
1212        {
1213          "exemplar": true,
1214          "expr": "label_replace(sum(irate(mongodb_wiredTiger_cache_bytes_read_into_cache{group_id=~\"$group_id\", cl_name=~\"$cl_name\", rs_nm=~\"$rs_nm\", instance=~\"$host.*\",  process_port=~\"$process_port\"}[$Interval])) by (instance, process_port) , \"hostname\", \"$1\", \"instance\", \"(.*)\")",
1215          "interval": "",
1216          "legendFormat": "{{hostname}}:{{process_port}}",
1217          "refId": "A",
1218          "datasource": {
1219            "type": "prometheus",
1220            "uid": "${DS_THANOS}"
1221          }
1222        }
1223      ],
1224      "thresholds": [],
1225      "timeRegions": [],
1226      "title": "Cache Activity - Read Into",
1227      "tooltip": {
1228        "shared": true,
1229        "sort": 0,
1230        "value_type": "individual"
1231      },
1232      "type": "graph",
1233      "xaxis": {
1234        "mode": "time",
1235        "show": true,
1236        "values": []
1237      },
1238      "yaxes": [
1239        {
1240          "format": "Bps",
1241          "logBase": 1,
1242          "show": true
1243        },
1244        {
1245          "format": "short",
1246          "logBase": 1,
1247          "show": true
1248        }
1249      ],
1250      "yaxis": {
1251        "align": false
1252      }
1253    },
1254    {
1255      "aliasColors": {},
1256      "bars": false,
1257      "dashLength": 10,
1258      "dashes": false,
1259      "datasource": {
1260        "type": "prometheus",
1261        "uid": "${DS_THANOS}"
1262      },
1263      "description": "The average rate of bytes per second written from WiredTiger's cache over the selected sample period.\n",
1264      "fieldConfig": {
1265        "defaults": {
1266          "links": [],
1267          "unit": "Bps"
1268        },
1269        "overrides": []
1270      },
1271      "fill": 1,
1272      "fillGradient": 0,
1273      "gridPos": {
1274        "h": 8,
1275        "w": 6,
1276        "x": 6,
1277        "y": 30
1278      },
1279      "hiddenSeries": false,
1280      "id": 78,
1281      "legend": {
1282        "avg": false,
1283        "current": false,
1284        "max": false,
1285        "min": false,
1286        "show": true,
1287        "total": false,
1288        "values": false
1289      },
1290      "lines": true,
1291      "linewidth": 1,
1292      "nullPointMode": "null",
1293      "options": {
1294        "alertThreshold": true,
1295        "legend": {
1296          "calcs": [],
1297          "displayMode": "list",
1298          "placement": "bottom"
1299        },
1300        "tooltip": {
1301          "mode": "single"
1302        }
1303      },
1304      "percentage": false,
1305      "pluginVersion": "8.5.6",
1306      "pointradius": 2,
1307      "points": false,
1308      "renderer": "flot",
1309      "seriesOverrides": [],
1310      "spaceLength": 10,
1311      "stack": false,
1312      "steppedLine": false,
1313      "targets": [
1314        {
1315          "exemplar": true,
1316          "expr": "label_replace(sum(irate(mongodb_wiredTiger_cache_bytes_written_from_cache{group_id=~\"$group_id\", cl_name=~\"$cl_name\", rs_nm=~\"$rs_nm\", instance=~\"$host.*\",  process_port=~\"$process_port\"}[$Interval])) by (instance, process_port) , \"hostname\", \"$1\", \"instance\", \"(.*)\")",
1317          "interval": "",
1318          "legendFormat": "{{hostname}}:{{process_port}}",
1319          "refId": "A",
1320          "datasource": {
1321            "type": "prometheus",
1322            "uid": "${DS_THANOS}"
1323          }
1324        }
1325      ],
1326      "thresholds": [],
1327      "timeRegions": [],
1328      "title": "Cache Activity - Write From",
1329      "tooltip": {
1330        "shared": true,
1331        "sort": 0,
1332        "value_type": "individual"
1333      },
1334      "type": "graph",
1335      "xaxis": {
1336        "mode": "time",
1337        "show": true,
1338        "values": []
1339      },
1340      "yaxes": [
1341        {
1342          "format": "Bps",
1343          "logBase": 1,
1344          "show": true
1345        },
1346        {
1347          "format": "short",
1348          "logBase": 1,
1349          "show": true
1350        }
1351      ],
1352      "yaxis": {
1353        "align": false
1354      }
1355    },
1356    {
1357      "aliasColors": {},
1358      "bars": false,
1359      "dashLength": 10,
1360      "dashes": false,
1361      "datasource": {
1362        "type": "prometheus",
1363        "uid": "${DS_THANOS}"
1364      },
1365      "description": "The number of bytes currently in the WiredTiger cache.",
1366      "fieldConfig": {
1367        "defaults": {
1368          "links": [],
1369          "unit": "bytes"
1370        },
1371        "overrides": []
1372      },
1373      "fill": 1,
1374      "fillGradient": 0,
1375      "gridPos": {
1376        "h": 8,
1377        "w": 6,
1378        "x": 12,
1379        "y": 30
1380      },
1381      "hiddenSeries": false,
1382      "id": 79,
1383      "legend": {
1384        "avg": false,
1385        "current": false,
1386        "max": false,
1387        "min": false,
1388        "show": true,
1389        "total": false,
1390        "values": false
1391      },
1392      "lines": true,
1393      "linewidth": 1,
1394      "nullPointMode": "null",
1395      "options": {
1396        "alertThreshold": true,
1397        "legend": {
1398          "calcs": [],
1399          "displayMode": "list",
1400          "placement": "bottom"
1401        },
1402        "tooltip": {
1403          "mode": "single"
1404        }
1405      },
1406      "percentage": false,
1407      "pluginVersion": "8.5.6",
1408      "pointradius": 2,
1409      "points": false,
1410      "renderer": "flot",
1411      "seriesOverrides": [],
1412      "spaceLength": 10,
1413      "stack": false,
1414      "steppedLine": false,
1415      "targets": [
1416        {
1417          "exemplar": true,
1418          "expr": "label_replace(sum(mongodb_wiredTiger_cache_bytes_currently_in_the_cache{group_id=~\"$group_id\", cl_name=~\"$cl_name\", rs_nm=~\"$rs_nm\", instance=~\"$host.*\",  process_port=~\"$process_port\"}) by (instance, process_port) , \"hostname\", \"$1\", \"instance\", \"(.*)\")",
1419          "interval": "",
1420          "legendFormat": "{{hostname}}:{{process_port}}",
1421          "refId": "A",
1422          "datasource": {
1423            "type": "prometheus",
1424            "uid": "${DS_THANOS}"
1425          }
1426        }
1427      ],
1428      "thresholds": [],
1429      "timeRegions": [],
1430      "title": "Cache Usage - Used",
1431      "tooltip": {
1432        "shared": true,
1433        "sort": 0,
1434        "value_type": "individual"
1435      },
1436      "type": "graph",
1437      "xaxis": {
1438        "mode": "time",
1439        "show": true,
1440        "values": []
1441      },
1442      "yaxes": [
1443        {
1444          "format": "bytes",
1445          "logBase": 1,
1446          "show": true
1447        },
1448        {
1449          "format": "short",
1450          "logBase": 1,
1451          "show": true
1452        }
1453      ],
1454      "yaxis": {
1455        "align": false
1456      }
1457    },
1458    {
1459      "aliasColors": {},
1460      "bars": false,
1461      "dashLength": 10,
1462      "dashes": false,
1463      "datasource": {
1464        "type": "prometheus",
1465        "uid": "${DS_THANOS}"
1466      },
1467      "description": "The number of tracked dirty bytes currently in the WiredTiger cache.",
1468      "fieldConfig": {
1469        "defaults": {
1470          "links": [],
1471          "unit": "bytes"
1472        },
1473        "overrides": []
1474      },
1475      "fill": 1,
1476      "fillGradient": 0,
1477      "gridPos": {
1478        "h": 8,
1479        "w": 6,
1480        "x": 18,
1481        "y": 30
1482      },
1483      "hiddenSeries": false,
1484      "id": 80,
1485      "legend": {
1486        "avg": false,
1487        "current": false,
1488        "max": false,
1489        "min": false,
1490        "show": true,
1491        "total": false,
1492        "values": false
1493      },
1494      "lines": true,
1495      "linewidth": 1,
1496      "nullPointMode": "null",
1497      "options": {
1498        "alertThreshold": true,
1499        "legend": {
1500          "calcs": [],
1501          "displayMode": "list",
1502          "placement": "bottom"
1503        },
1504        "tooltip": {
1505          "mode": "single"
1506        }
1507      },
1508      "percentage": false,
1509      "pluginVersion": "8.5.6",
1510      "pointradius": 2,
1511      "points": false,
1512      "renderer": "flot",
1513      "seriesOverrides": [],
1514      "spaceLength": 10,
1515      "stack": false,
1516      "steppedLine": false,
1517      "targets": [
1518        {
1519          "exemplar": true,
1520          "expr": "label_replace(sum(mongodb_wiredTiger_cache_tracked_dirty_bytes_in_the_cache{group_id=~\"$group_id\", cl_name=~\"$cl_name\", rs_nm=~\"$rs_nm\", instance=~\"$host.*\",  process_port=~\"$process_port\"}) by (instance, process_port) , \"hostname\", \"$1\", \"instance\", \"(.*)\")",
1521          "interval": "",
1522          "legendFormat": "{{hostname}}:{{process_port}}",
1523          "refId": "A",
1524          "datasource": {
1525            "type": "prometheus",
1526            "uid": "${DS_THANOS}"
1527          }
1528        }
1529      ],
1530      "thresholds": [],
1531      "timeRegions": [],
1532      "title": "Cache Usage - Dirty",
1533      "tooltip": {
1534        "shared": true,
1535        "sort": 0,
1536        "value_type": "individual"
1537      },
1538      "type": "graph",
1539      "xaxis": {
1540        "mode": "time",
1541        "show": true,
1542        "values": []
1543      },
1544      "yaxes": [
1545        {
1546          "format": "bytes",
1547          "logBase": 1,
1548          "show": true
1549        },
1550        {
1551          "format": "short",
1552          "logBase": 1,
1553          "show": true
1554        }
1555      ],
1556      "yaxis": {
1557        "align": false
1558      }
1559    },
1560    {
1561      "collapsed": false,
1562      "datasource": {
1563        "type": "prometheus",
1564        "uid": "000000010"
1565      },
1566      "gridPos": {
1567        "h": 1,
1568        "w": 24,
1569        "x": 0,
1570        "y": 38
1571      },
1572      "id": 28,
1573      "panels": [],
1574      "title": "Connections",
1575      "type": "row"
1576    },
1577    {
1578      "aliasColors": {},
1579      "bars": false,
1580      "dashLength": 10,
1581      "dashes": false,
1582

Prerequisites

Procedure

Navigate to the .css-h15tq0{font-style:normal;font-weight:700;}Integrations page for your project.

Click Configure for the Prometheus integration card.

Enter your preferred username and password.

Important

Enter your IP address and port.

Tip

(Optional) Encrypt all Prometheus metrics.

Select your preferred service discovery method.

Click Save.

View Your Cluster Metrics on Prometheus.

Example Configurations

HTTP Service Discovery

File Service Discovery

Import a Sample Grafana Dashboard

Navigate to the Import screen.

Click Upload JSON File.

Copy over the mongo-metrics.json or the hardware-metrics.json file to create a dashboard.

mongo-metrics.json

Navigate to the Integrations page for your project.

Copy over the `mongo-metrics.json` or the `hardware-metrics.json` file to create a dashboard.