> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mirador.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Metric Alerts

> PromQL threshold alerts with hysteresis, and the notification channels they deliver to

A metric alert is a PromQL threshold that notifies a channel when it crosses. It's the alerting twin of a [dashboard](/api/dashboards): the **same** slug identity, the same `If-None-Match`/`If-Match` ETag contract, the same `428`/`412` rules — only the collection name and body differ.

## Reading alerts

Reads are always safe:

* `GET /v1/metric-alerts` — summaries, with a cache-only collection ETag for `If-None-Match`.
* `GET /v1/metric-alerts/{slug}` — one alert as a bare object, plus its resource ETag (an HTTP header) for conditional writes.

## Writing alerts

<Warning>
  An enabled alert **pages people and fires webhooks**. A server key can create,
  replace, and delete every alert in its project, and writes are not attributed.
  `GET` and keep the current body before any replace or delete, and don't let an
  automation or agent mutate alerts unless that's explicitly its job.
</Warning>

### The document

The exact contract is `MetricAlertInput` in the [OpenAPI spec](https://api.mirador.org/openapi.yaml); the essentials:

* `display_name` — required, 1–255 characters. The `slug` is the immutable URL identity, never in the body.
* `severity` — required; one of `info`, `warning`, `critical`.
* `enabled` — required boolean, honored atomically: `true` starts evaluating immediately, and **omitting the key is a `400`**, not a silent disable.
* `condition.expression` — required PromQL: a **top-level comparison** of a vector query against **one finite number literal** (`>`, `>=`, `<`, `<=`), e.g. `sum(rate(http_requests_total{"http.response.status_code"=~"5.."}[5m])) > 5`. Every selector needs an exact metric name.
* `condition.recovery_expression` — optional hysteresis: a second comparison pointing the opposite way whose band must not overlap the firing band (firing `> 80`, recovery `< 70`). Omitted → recovery is the negation of the firing expression.
* `notifications` — required array of **integration channel slugs** (discover them via [`/v1/integrations`](#notification-channels)). It's a set — order doesn't matter; a duplicate or unknown slug is `400`; `[]` is valid (the alert fires but notifies nobody).
* `documentation.markdown` — optional runbook (up to 16,384 bytes), delivered verbatim into every notification for this alert. Use inline links: `[Runbook](https://…)`.
* Unknown JSON fields are rejected with `400`; the body is capped at 1 MiB; `Content-Type: application/json` is required.

### Grouped alerts: one incident per series

The condition may return **one series or many**. A bare aggregate (`sum(...)`) is one series and fires one incident. A **grouped** query — `sum by ("chain") (rate(m[5m])) > 5` — fires **one independent incident per dimension value**: `chain="ethereum"` can fire while `chain="bsc"` stays healthy, and the notification names the dimension that broke. Grouping is authored **only** inside the PromQL `by (...)` clause — there is no separate group-by field.

### Sustained-ness: there is no `for`

Express duration inside the window function, and pick the function to match the intent:

| Intent                                                     | Expression                   |
| ---------------------------------------------------------- | ---------------------------- |
| Fire on the window *average* (a brief dip doesn't stop it) | `avg_over_time(m[5m]) > 0.9` |
| Classic "stayed above X for the whole window" (`for: 5m`)  | `min_over_time(m[5m]) > X`   |
| "Stayed below Y for the whole window"                      | `max_over_time(m[5m]) < Y`   |

Firing and recovery may use different window functions as long as their numeric bands don't overlap — e.g. firing `min_over_time(m[5m]) > 600` with recovery `max_over_time(m[5m]) < 60` gives a dead-band the signal must fully clear, so the alert can't flap around a single threshold.

### Execution semantics

Evaluation is **event-driven**, deliberately not Prometheus-compatible: each incoming datapoint for a watched metric re-evaluates the condition. A no-data, non-finite, or query-error evaluation **holds** the current state and never fires. Each returned series is its own incident keyed on its label set, opened and resolved independently.

### Create

The conditional-write contract is identical to [dashboards](/api/dashboards#the-conditional-write-contract): create with `If-None-Match: *`, replace/delete with `If-Match: "<etag>"` from a fresh `GET`.

```bash theme={null}
# 1. Find the channel slug to notify.
curl -sS --fail-with-body \
  -H "Authorization: Bearer $MIRADOR_SERVER_KEY" \
  https://api.mirador.org/v1/integrations

# 2. Author alert.json, then create it.
cat > alert.json <<'JSON'
{
  "display_name": "Checkout 5xx error rate",
  "severity": "critical",
  "enabled": true,
  "condition": {
    "expression": "sum(rate(http_requests_total{\"service.name\"=\"checkout\",\"http.response.status_code\"=~\"5..\"}[5m])) > 5"
  },
  "notifications": ["checkout-oncall"],
  "documentation": { "markdown": "A 5xx spike usually means a bad deploy. Roll back, then check DB latency." }
}
JSON

curl -sS --fail-with-body -X PUT \
  -H "Authorization: Bearer $MIRADOR_SERVER_KEY" \
  -H 'Content-Type: application/json' \
  -H 'If-None-Match: *' \
  --data-binary @alert.json \
  https://api.mirador.org/v1/metric-alerts/checkout-5xx
```

## Notification channels

`GET /v1/integrations` and `GET /v1/integrations/{slug}` list the notification channels an alert can deliver to, as `{slug, display_name, type, enabled}` — **redacted**: no webhook URL, headers, or routing keys are ever returned.

* Channels are **organization-scoped** — the one collection that isn't per-project. One slug is shared by every project in the org: an alert in project A and one in project B can both notify the channel named `oncall`.
* This surface is **read-only**. Create and configure channels in the Mirador web app (see [Integrations](/automations/integrations)), then reference them by slug in an alert's `notifications`.

## Next Steps

<CardGroup cols={2}>
  <Card title="Metrics" icon="chart-line" href="/api/metrics">
    Discover the metrics your conditions evaluate
  </Card>

  <Card title="Automations" icon="bolt" href="/automations/overview">
    Trace-based rules and integrations in the web app
  </Card>
</CardGroup>