Monitoring a database is important to identify performance bottlenecks and areas for optimization. Currently HarperDB exposes its metrics via the `system_information` endpoint and visualizes them on HarperDB Studio under the `status` tab:
However, if you wish to export these metrics to your existing monitoring stack, currently there are no native plugins for popular monitoring tools including Prometheus, Datadog, or Splunk. In this post, we’ll explore how to export HarperDB metrics to Prometheus, utilizing the Push Gateway to periodically update metrics.
First, we will run a local instance of HarperDB with default configurations:
Once the database is ready, we can check for the system information:
NOTE: the response format differs slightly from the API docs so make sure to use the format returned by the version of HarperDB you are running locally.
We are now ready to also deploy Prometheus locally for testing. First, we need to add a configuration for Prometheus server to also scrape Push Gateway.
Create a new directory called `prometheus` and add a new file called `prometheus.yml` with the following contents:
Then we can create our docker compose file:
Here we are mounting the prometheus config file to the server and running both server and pushgateway containers with ports mapped to localhost.
You can navigate to http://localhost:9090/targets to verify that both components are up and running:
Exporting Metrics to Prometheus
Finally, we’re ready to start exporting metrics to Prometheus. First, we need to store our basic auth token to `.env`. If you used the above docker command to run HarperDB, you can copy the BASIC_AUTH variable and paste it to `.env`:
Then create `index.js` with the following contents:
Let’s take a deeper look at this code:
- We are registering two metrics `cpuLoad` and `usedMemory` and creating a Prometheus Gauge.
- Every 5 seconds, we poll the system information from HarperDB and we set the gauge values accordingly.
- Finally, we publish those metrics to Push Gateway via the `gateway.push` method.
- These metrics are registered to the Push Gateway’s register, which is in turn scraped by Prometheus server as specified in the config file we mounted earlier.
Run this code and you should see log messages like:
Navigate to localhost:9090/graph to start querying our data.
For example, for CPU load, we can search for `harperdb_cpu_load`:
Likewise, we can also graph memory usage:
In this post, we saw how we can use the Prometheus client library to export HarperDB metrics available via its system_information endpoint. We registered two example CPU load and memory usage metrics. You can easily extend this example with other metrics available in HarperDB such as disk information or more granular CPU/memory statistics.