Skip to content
Optimal Lab

DevOps · interactive

Prometheus metrics: counters, rates and histograms

Prometheus scrapes numbers from your app every 15 seconds. A request counter only ever goes up, until the pod restarts and it drops to zero. Raw counters are almost useless; rate() and histograms turn them into answers.

http_requests_total
0 srestart at 150 s300 s

A total since the process started. It climbs forever, and drops to 0 when the pod restarts at 150 s. The height tells you nothing about current traffic.

Query

Step 1 of 4

  1. 1

    Counters only go up

    Requests, errors, bytes. Always query them through rate() or increase().

    Gauges (memory, queue length) go up and down.

  2. 2

    rate() handles resets

    Per-second average increase over the window. Use a window at least 4× the scrape interval.

    sum by (route) (rate(http_requests_total[5m]))

  3. 3

    Histograms for percentiles

    Averages hide slow requests. Buckets let you compute p95 and p99 across all pods.

    Can't average percentiles; can sum buckets.

  4. 4

    Watch label cardinality

    Every label combination is a separate series. Never label by user id or full URL.

    Millions of series will sink Prometheus.

In production

Tools and technologies

Prometheus is the default metrics system for cloud-native apps.

Not tied to one language — use these names when you search docs or design a system.