Sauron

One eye on everything you self-host. Sauron collects the notifications your own infrastructure produces, in one place, as a readable timestamped history per service.

Backup scripts, n8n flows, release watchers and cron wrappers all have something to say, and by default they say it over e-mail or not at all. Mail is a poor fit: it has no severity, no history you can scan, no way to tell that a job has simply stopped reporting, and it buries a "backup completed" among everything else. Sauron gives each source its own feed, gives every message a severity, and forwards onward only what you actually asked to be told about.

What it does

• A feed per service, newest first, with Markdown formatting and embedded images.
• Severity levels: the eight syslog levels plus Success, and any custom level you add, each with its own colour.
• Live updates over server-sent events, so an open feed shows a notification the moment it arrives.
• Token authentication per service, created in the interface and revocable, so a compromised script cannot post as anything else.
• Onward delivery by SMTP, SparkPost or webhook, opted into per service and filtered to the levels you choose, so nightly successes stay quiet while failures reach you.
• Single sign-on through any OpenID Connect provider, with roles resolved from group membership.
• A Prometheus exporter, so Grafana can chart activity and alert on a service that has gone quiet.

Getting started

Sauron runs behind your existing reverse proxy, which terminates TLS. It needs no database container: state is a single SQLite file plus a directory of stored images, bind mounted from the host.

git clone ssh://git@forge.quinlan.cloud:2222/joshquinlan/sauron.git

cp .env.example .env

Fill in .env. At a minimum you need SAURON_BASE_URL, a session key, a metrics token and your identity provider details. Generate the two secrets with:

openssl rand -base64 48

The container runs unprivileged as uid 65532, and Docker does not change the ownership of a bind mount, so create the data directory with that owner before the first start or Sauron cannot write its database:

sudo install -d -o 65532 -g 65532 ./data

Then bring it up:

docker compose up -d --build

The compose file joins the external citadel-routing network and publishes no host ports, so the reverse proxy reaches Sauron at sauron:8080 over that network and there is no second, unproxied way in. Make sure the public URL your proxy serves matches SAURON_BASE_URL, because that is what OAuth redirects and the image links in outgoing notifications are built from.

Set SAURON_DATA_PATH if you want the data somewhere other than ./data.

To stamp the build with a real version rather than dev:

SAURON_VERSION=$(git describe --tags --always) docker compose up -d --build

The build stage runs on the host network so it can reach the module proxy from behind whatever the host uses to get out.

Setting up the identity provider

Create an OAuth2/OpenID Connect provider in Authentik (or any other IdP) with:

• Redirect URI: https://sauron.example.com/auth/callback
• Scopes: openid, profile, email, and a scope that emits a groups claim

Then create three groups and map them with SAURON_GROUP_ADMIN, SAURON_GROUP_OPERATOR and SAURON_GROUP_VIEWER. A user who is in none of them is refused unless you set a default role.

Role	Can do
Viewer	Read the dashboard and every service feed
Operator	Also manage services, tokens and per-service routing
Admin	Also manage providers, custom levels and global settings

Roles are re-read from the groups claim at every sign-in, so removing someone from a group takes effect the next time they log in rather than whenever their session happens to expire.

Sending notifications

Add a service in Services & tokens, create a token, and post to /api/v1/notify. The token goes in the Authorization header.

curl -X POST https://sauron.example.com/api/v1/notify \
  -H "Authorization: Bearer $SAURON_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"level":"success","title":"Snapshot complete","body":"Backed up **4 repositories**."}'

Every field except one of title or body is optional:

Field	Meaning
title	Short headline shown in bold
body	Markdown, rendered and sanitised
level	Level address, for example error. Defaults to info
link	URL shown as a link under the message
image_url	Image to embed; copied to local storage at ingest
image_urls	Several images, as an array
metadata	Object of string keys and values, shown as chips
timestamp	RFC 3339 time, if the event is older than the request

message is accepted as a synonym for body.

For shell scripts that should not have to build JSON, post the body as plain text and pass the level in the query string:

curl -X POST "https://sauron.example.com/api/v1/notify?level=error&title=Backup%20failed" \
  -H "Authorization: Bearer $SAURON_TOKEN" \
  -H "Content-Type: text/plain" \
  --data "restic exited 11: repository is locked"

To attach a file from disk, post it as multipart. The text fields keep the same names, and metadata uses a metadata. prefix:

curl -X POST https://sauron.example.com/api/v1/notify \
  -H "Authorization: Bearer $SAURON_TOKEN" \
  -F "level=info" \
  -F "title=Weekly capacity report" \
  -F "body=Usage is trending **upward**." \
  -F "metadata.window=7d" \
  -F "chart=@chart.png;type=image/png"

A successful post returns 201 with the notification id. A bad token returns 401, an unknown level or an empty notification 400.

Images given as a URL are fetched and stored locally at ingest, so an embed still renders after the source has gone. The fetcher refuses loopback, link-local and private addresses, and re-checks after every redirect, so a token cannot be used to probe the network Sauron sits inside.

Severity levels

Level	Rank	Level	Rank
Debug	0	Error	40
Info	10	Critical	50
Success	15	Alert	60
Notice	20	Emergency	70
Warning	30

Rank orders the levels; higher is more severe. Built-in ranks are spaced ten apart so a custom level slots between them, and the dashboard counts Warning and above as a problem. Built-in levels can be recoloured and renamed but keep their address, because that is what services send and what Prometheus series are keyed by.

Onward delivery

Configure providers once under Notification providers, then opt in per service under that service's settings. Each route picks an explicit set of levels rather than a threshold, so "tell me about Success and Error but not Warning" is expressible. A route can override the provider's recipient, letting one SMTP server serve several addresses.

Webhook payloads are signed with HMAC-SHA256 in the X-Sauron-Signature header when a secret is set. Failed deliveries retry with bounded backoff and every attempt is recorded, so a broken provider shows up as failures rather than as silence.

Prometheus and Grafana

/metrics requires the bearer token from SAURON_METRICS_TOKEN:

scrape_configs:
  - job_name: sauron
    static_configs:
      - targets: ["sauron:8080"]
    authorization:
      credentials: "the value of SAURON_METRICS_TOKEN"

Metric	Meaning
sauron_notifications_total{service,level}	Stored total, read from the database so it survives restarts
sauron_notifications_received_total{service,level}	Counter for this process
sauron_service_last_notification_timestamp_seconds{service}	Alert on a service that has gone quiet
sauron_service_quiet_after_seconds{service}	That service's own silence threshold
sauron_ingest_rejected_total{reason}	Bad tokens, unknown levels, malformed bodies
sauron_deliveries_total{provider,status}	Onward delivery outcomes
sauron_deliveries_pending, sauron_deliveries_failed	Delivery backlog
sauron_services, sauron_users	Current counts
sauron_build_info{version}	Version stamped at build time

Because each service exports its own threshold, one alert covers all of them and respects whatever you configured per service:

time() - sauron_service_last_notification_timestamp_seconds
  > sauron_service_quiet_after_seconds > 0

Noticing silence

Silence is often the interesting signal: a backup job that stops reporting looks exactly like one that has nothing to say. The dashboard flags a service that has said nothing for longer than its threshold, which defaults to SAURON_QUIET_AFTER (48 hours) and can be overridden per service, because a five-minute health check and a weekly report mean very different things by "quiet". A service set to zero is never flagged, and one that has never reported at all is not flagged either, since it has not gone quiet so much as never started.

Retention

Notifications are kept for SAURON_RETENTION_DAYS (90 by default). Any service can override this in its own settings, and zero keeps them forever. A background pass prunes expired notifications, expired sessions, and stored images no longer referenced by anything. Images are content-addressed, so the same picture sent twice is stored once and is only deleted once nothing refers to it.

Configuration

Everything is read from the environment at startup. Invalid values are all reported at once rather than one restart at a time.

Variable	Default	Meaning
SAURON_BASE_URL	required	Externally reachable URL
SAURON_SESSION_KEY	required	Session secret, 32+ characters
SAURON_METRICS_TOKEN	required	Bearer token for /metrics
SAURON_ADDR	:8080	Listen address
SAURON_DATA_DIR	/data	Database and stored images
SAURON_DB_PATH	$DATA_DIR/sauron.db	Database file
SAURON_MEDIA_DIR	$DATA_DIR/media	Stored images
SAURON_LOG_LEVEL	info	debug, info, warn, error
SAURON_OIDC_ISSUER		Provider issuer URL
SAURON_OIDC_CLIENT_ID		Required when an issuer is set
SAURON_OIDC_CLIENT_SECRET		Required when an issuer is set
SAURON_OIDC_SCOPES	openid,profile,email,groups	Requested scopes
SAURON_OIDC_GROUPS_CLAIM	groups	Claim holding group names
SAURON_GROUP_ADMIN	sauron-admins	Group granting admin
SAURON_GROUP_OPERATOR	sauron-operators	Group granting operator
SAURON_GROUP_VIEWER	sauron-viewers	Group granting viewer
SAURON_GROUP_DEFAULT_ROLE		Role when no group matches; blank denies
SAURON_SESSION_TTL	720h	Session lifetime
SAURON_SECURE_COOKIE	true	Set false only without HTTPS
SAURON_RETENTION_DAYS	90	Global retention; zero keeps forever
SAURON_QUIET_AFTER	48h	Default silence before a service is flagged
SAURON_RETENTION_INTERVAL	1h	How often the pruner runs
SAURON_METRICS_ENABLED	true	Serve /metrics at all
SAURON_MEDIA_MAX_BYTES	16mb	Largest image accepted
SAURON_MEDIA_FETCH_TIMEOUT	15s	Remote image fetch timeout
SAURON_MEDIA_FETCH_REMOTE	true	Allow image_url fetching
SAURON_MEDIA_ALLOW_UPLOADS	true	Allow multipart uploads
SAURON_MEDIA_ALLOWED_TYPES	image/png,image/jpeg,image/gif,image/webp	Accepted image types
SAURON_DEV_LOGIN	false	Local development only; see below

Development

Sauron is a single Go binary with no build step for the frontend: templates, stylesheet and htmx are embedded in it.

go test ./...

To run it locally without an identity provider, set SAURON_DEV_LOGIN=true and leave SAURON_OIDC_ISSUER empty. Every visitor is then signed in as an administrator, and the server logs a warning at startup saying so. Sauron refuses to start if neither an issuer nor dev login is configured, so this cannot be reached by accident in production.

SAURON_BASE_URL=http://localhost:8080 \
SAURON_SESSION_KEY=$(openssl rand -base64 48) \
SAURON_METRICS_TOKEN=$(openssl rand -hex 32) \
SAURON_DEV_LOGIN=true \
SAURON_SECURE_COOKIE=false \
SAURON_DATA_DIR=./data \
go run ./cmd/sauron

The layout follows the usual Go shape: cmd/sauron is the entrypoint, and each package under internal owns one concern — store for the database, web for the interface and ingest API, notify for onward delivery, media for images, auth for sign-in, metrics for the exporter.

Database changes are additive migrations in internal/db/migrations, applied automatically at startup and recorded so they run once.