Oculo documentation

Oculo is a self-hosted platform for logs and errors. Your apps send log events, and Oculo stores them, makes them searchable, groups errors into issues, and alerts you when something breaks. People use the web dashboard. Scripts and LLM agents (Claude, ChatGPT, Cursor…) use the same data through a REST API, a CLI and an MCP server.

These docs are written for both people and LLMs. A running server shows all of them as one page at /docs, serves the same content as raw Markdown at /docs.md, and has a shorter agent guide at /llms.txt.

#Quick start

1. Start the server (Node 20+):
   Copy

   npm install
   npm run dev          # https://oculo.funonymous.in

2. Create the first account. Open the dashboard. The first account becomes the instance admin. (An admin API key is also printed on first start and saved to server/data/admin-key.txt.)
3. Create a project on the Projects page. You get an ingest key (to send logs) and a read key (to query, which is safe to give to an LLM).
4. Send a log:
   Copy

   curl -X POST https://oculo.funonymous.in/api/v1/ingest \
     -H "Authorization: Bearer oc_ingest_…" -H "Content-Type: application/json" \
     -d '[{"level":"error","message":"Payment failed","service":"api","attributes":{"orderId":8812}}]'

5. Look at it. It shows up within a second on Logs and Live, and as an Issue because it's an error.

For demo data, run npm run seed while the server is running. It loads about 24 hours of realistic logs across two releases.

Next: see everything Oculo can do in Features, run it for real with Deployment, and when volume grows, read Scaling.

#Features

A tour of everything Oculo does. Each item links to the page with the details.

#Collect

• One HTTP endpoint takes JSON arrays, single objects or NDJSON, optionally gzip/deflate/brotli compressed. Common field aliases (msg, severity, @timestamp, traceId…) are understood and unknown fields are kept as attributes. See Sending logs.
• JavaScript/TypeScript SDK for Node and browsers: batching, retries, flush on exit, uncaught-error capture, Express middleware, per-request breadcrumbs, console capture, source-map upload.
• Python SDK with no dependencies: stdlib logging bridge, ASGI (FastAPI/Starlette) and Django middleware, per-request breadcrumbs.
• OpenTelemetry: any OTLP/HTTP log exporter (protobuf or JSON) works, so Go, Java, .NET, Rust and others need no Oculo SDK.
• Heartbeats for cron jobs: one curl at the end of the job; a missed check-in alerts you.
• Privacy at ingest: passwords, tokens, cookies, auth headers, JWTs and card numbers are scrubbed on the server; email masking and extra keys are per project.
• Sampling and drop rules per project (for example keep 10% of debug logs, drop health checks). Errors are never sampled away.

#Find

• One query language everywhere (dashboard, API, CLI, MCP, alerts): free text, phrases, exclusions, field:value, comparisons, wildcards, nested and dotted attributes, has:, time windows. See Query language.
• Logs page: autocomplete, filter chips, facets with only / exclude, a level-stacked volume chart you can drag to zoom, saved and recent searches, NDJSON/CSV export, shareable URLs.
• Log detail: every field with filter/exclude/copy, source-mapped stack trace, attributes, breadcrumbs, surrounding logs, the whole trace, "Copy for LLM".
• Live tail with your filter applied, pausable, reconnecting.
• Log patterns: millions of similar lines collapse into templates like Charging order <*>, with counts and trends. See Log patterns.
• Metrics from logs: count, avg, sum, min, max, p50/p90/p95/p99 of any numeric attribute, over time, grouped by any field. No separate metrics pipeline. See Metrics.
• Traces: every log sharing a trace_id, in order, across services.

#Understand errors

• Issues: errors are fingerprinted (exception type, message with variables stripped, top in-app frames) so a thousand occurrences are one issue with a count, affected users, releases and a 48-hour/30-day timeline. See Issues.
• Regressions: a resolved issue that happens again reopens and is flagged.
• Custom grouping with a fingerprint field.
• Source maps: minified JavaScript stacks are rewritten to original files, lines and functions before grouping. See Releases and source maps.
• Releases: error rate per release, deploy markers on charts, and "what changed" between two releases (new, increased, gone issues).
• ✨ Explain (opt-in): sends one issue to Claude for a likely cause and fix, with consent and caching.

#Get told

• Alert types: threshold, spike (vs. a 12-window baseline), new issue, regression, missed heartbeat, daily digest. See Alerts and heartbeats.
• Channels: Slack, Discord, email (SMTP), Telegram, PagerDuty (with auto-resolve), any JSON webhook.
• Cooldowns, test sends and delivery history; secrets are masked when read back.

#Work with LLMs and scripts

• MCP server with tools such as project_summary, search_logs, get_issue, query_metrics and explain_issue; read-only unless you opt in. See Using Oculo from an LLM.
• Agent-friendly REST: format=md for compact tables, fields= to trim rows, cursors, hard limits, /llms.txt, /openapi.json, /docs.md.
• CLI: oculo search, tail, issues, summary, sourcemaps upload, heartbeat, with --json for piping. See CLI.

#Run it your way

• Single process, single file by default: Node 20 + SQLite (WAL, FTS5). npm install && npm run dev and you're running.
• Scale out when needed: store logs in ClickHouse and put Redis Streams in front of ingest by setting two environment variables. Search, issues, alerts and the API behave the same. See Scaling.
• Teams: email/password accounts and GitHub sign-in, invite links, per-project roles (viewer, member, owner) and instance admins; scoped API keys for apps and agents. See Concepts.
• Retention per project, with hourly rollups and issue timelines kept much longer.
• Docker Compose file for Oculo + ClickHouse + Redis. See Deployment.

#Concepts

#Levels

trace < debug < info < warn < error < fatal. Common aliases are understood: warning, err, critical, pino numbers 10–60, and OTel severity numbers. Anything at error or above, or carrying an exception, becomes part of an issue.

#Keys, accounts and roles

There are two ways in, and both end up with a role on a project.

• viewer: read everything in the project.
• member: also resolve, ignore and assign issues, save searches, manage alerts and heartbeats, and upload source maps.
• owner: also change project settings, manage keys and people, and delete the project.
• admin (instance admin, such as the first account or an admin key): everything, in every project, including creating projects.

Send keys as Authorization: Bearer <key>. The X-Oculo-Key header and the ?key= query parameter also work. The dashboard uses an HttpOnly session cookie instead of keys when you sign in with an account.

#Sending logs

Every SDK is a thin wrapper around one endpoint, so a language without an SDK can still send logs over plain HTTP.

#HTTP (any language)

POST /api/v1/ingest with an ingest key. The body can be a JSON array of events, a single object, {"events": […]}, or NDJSON. It may be gzip, deflate or br encoded. The limits are 5,000 events and 10 MB (decompressed) per request.

{
  "timestamp": "2026-11-14T16:20:01.123Z",
  "level": "error",
  "message": "Payment failed for order 8812",
  "logger": "billing.stripe",
  "service": "api", "environment": "prod", "release": "1.4.2", "host": "web-3",
  "trace_id": "abc123", "span_id": "def456",
  "user": { "id": "u_42", "email": "…" },
  "attributes": { "orderId": 8812, "amount": 499, "route": "/checkout" },
  "exception": { "type": "CardError", "value": "card_declined", "stacktrace": "…" },
  "fingerprint": ["optional", "custom", "grouping"]
}

• Only message is really needed; everything else is optional.
• Nothing is lost: unknown top-level fields move into attributes.
• Aliases work: msg, severity, @timestamp, env, traceId and similar names are understood.
• Timestamps: ISO strings, or epoch seconds, ms, µs or ns. Timestamps more than 5 minutes in the future are clamped to now.
• Custom grouping: fingerprint overrides issue grouping. Use "{{ default }}" inside it to extend the default fingerprint instead of replacing it.

The response is 202 {"accepted": n, "dropped": m}. Events are dropped only by the project's sampling and drop rules. With the Redis queue on, the response is 202 {"accepted": n, "dropped": 0, "queued": true} (rules run later), and a full queue answers 503 with Retry-After; see Scaling.

#JavaScript / TypeScript

import { Oculo } from '@gouravsrf/oculo';   // npm install @gouravsrf/oculo  (Node 18+ and browsers, ESM + CommonJS)

const log = Oculo.init({ key: 'oc_ingest_…', url: 'https://oculo.funonymous.in', env: 'prod', release: '1.4.2', service: 'api' });

log.info('User signed in', { userId: 42 });
log.error('Payment failed', { orderId }, err);
log.captureException(err);
const db = log.child({ logger: 'db' });          // child logger with extra context
log.setUser({ id: 'u_42' }); log.setTag('region', 'ap-south');

// Express: request logs, a request id + trace id on every log, and per-request breadcrumbs
app.use(log.expressMiddleware());
app.use(log.expressErrorHandler());

// Any other code path: give a unit of work its own breadcrumbs + context
await log.withScope(() => handleJob(job), { job_id: job.id });

• Batching: sends every 2 s or every 100 events, gzipped, with exponential backoff that honours Retry-After. The buffer is bounded at 1,000 events and drops the oldest when full.
• Never in your way: it never throws into your code and never blocks it.
• Flushes on exit, and through sendBeacon when a browser tab is hidden.
• Automatic capture: uncaught exceptions (it still exits with code 1), unhandled rejections, window.onerror, and optionally console.* (captureConsole: true | 'breadcrumbs').
• Breadcrumbs: the last 50 lower-level logs are attached to each error. With the middleware or withScope they are per request, so concurrent requests never mix.
• Filtering: beforeSend(event) lets you edit or drop events, and sampleRate keeps a fraction of non-error events.
• Source maps: Oculo.uploadSourceMap({url, key, release, file, map}) uploads one; see Releases and source maps.

#Python

import oculo                     # pip install /path/to/oculo/sdk-python  (3.8+, standard library only)

log = oculo.init(key="oc_ingest_…", url="https://oculo.funonymous.in", env="prod", release="1.4.2", service="worker")
log.info("User signed in", user_id=42)
try:
    charge()
except Exception:
    log.exception("Payment failed", order_id=8812)

import logging
logging.getLogger().addHandler(oculo.OculoHandler())        # bridge the stdlib logging module

with oculo.scope(request_id="r1", trace_id="abc"):          # per-request breadcrumbs + context (thread/asyncio safe)
    handle()

# FastAPI / Starlette:  app.add_middleware(oculo.middleware.OculoASGIMiddleware)
# Django:               MIDDLEWARE = ["oculo.middleware.OculoDjangoMiddleware", ...]

It behaves like the JS SDK: a background sender thread, batching, gzip, retries, flush at exit, and capture through sys.excepthook and threading.excepthook.

#OpenTelemetry (Go, Java, .NET, Rust, …)

Point any OTLP log exporter at Oculo. Both protobuf (the default) and JSON are accepted:

OTEL_EXPORTER_OTLP_LOGS_ENDPOINT=https://oculo.funonymous.in/v1/logs
OTEL_EXPORTER_OTLP_LOGS_PROTOCOL=http/protobuf        # or http/json
OTEL_EXPORTER_OTLP_LOGS_HEADERS=Authorization=Bearer%20oc_ingest_…

Resource attributes map to service (service.name), release (service.version), environment (deployment.environment.name) and host (host.name). Trace and span ids become hex, and exception.* attributes become the exception.

#Heartbeats from cron jobs

# at the end of the job (creates the heartbeat on first ping when ?period is given)
curl -fsS -X POST "https://oculo.funonymous.in/api/v1/heartbeats/nightly-backup/ping?period=1d" -H "Authorization: Bearer oc_ingest_…"
# or with the CLI
oculo heartbeat nightly-backup

#Searching: the query language

One search bar, used everywhere: the dashboard, the API (q=), the CLI, MCP tools and alert queries.

payment failed level:>=warn env:prod,staging -healthcheck
@status:>=500 @route:/checkout* release:1.4* trace:abc123 user:u_42
"connection refused" has:exception since:2h until:30m

Built-in fields:

• level (supports >=warn)
• env/environment, service, release, host, logger
• trace, span, user
• type (exception type), issue (issue id), pattern (pattern id), id, message

Free text is case- and accent-insensitive (cafe finds "Café"). Queries return the same results whether logs are stored in SQLite or ClickHouse (see Scaling).

Every value is a bound SQL parameter or a quoted full-text phrase, so queries can't inject anything. Malformed queries return 400 with a hint.

#The dashboard

• Overview: the project's health for 24h or 7d.
  • Headline numbers: logs, errors, error rate and open issues, each compared with the previous window.
  • A volume chart by level with deploy markers.
  • Top issues, issues needing attention (new or regressed), top error messages grouped by template, and the latest releases.
• Logs: search with autocomplete. An empty box offers your saved and recent searches.
  • Active filters show as chips. The sidebar facets toggle on and off, with only and exclude actions.
  • Time presets, a custom range and auto-refresh.
  • The chart shows volume stacked by level: drag across it to zoom, or click the legend to filter.
  • A Logs / Patterns switch.
  • Click a row for the detail drawer: fields with filter, exclude and copy actions, the stack trace (source-mapped when possible), attributes, breadcrumbs, surrounding logs, the trace, Copy for LLM and Copy JSON.
  • Export to NDJSON or CSV.
• Live: a real-time stream with your filter applied. Pause with space; events that arrive while paused are buffered.
• Issues: errors grouped by fingerprint, with sparklines and new/regressed/resolved badges. Select several to resolve, ignore or reopen them at once.
  • The issue page shows a timeline, the stack trace, breadcrumbs, attributes, tags, releases, affected users and recent events.
  • It has ✨ Explain (when enabled), Copy for LLM, resolve (r) and assign.
• Metrics: charts computed from logs. See Metrics.
• Releases: error rate per release, and a comparison of two releases (new, increased and gone issues).
• Alerts: alert rules, their history, and Heartbeats.
• Projects: create projects and get keys and setup snippets.
• Settings: the project, retention, privacy (email masking, extra sensitive keys), sampling and drop rules, the Team panel (members, roles, invite links), API keys, saved searches and project deletion.

Keyboard shortcuts:

• Ctrl/⌘ K opens the command palette.
• / focuses search.
• j / k move between rows; Enter opens the selected one.
• g then o, l, t, i, m or r jumps to a page.
• ? shows help.

The query and time range live in the URL, so any view can be shared as a link. There are dark and light themes, relative or absolute times in your local timezone, and the layout works on phones.

#Issues and grouping

Each error gets a fingerprint built from:

• the exception type
• the message with variable parts stripped (UUIDs, emails, URLs, IPs, dates, hex ids, quoted strings and numbers, including 3036ms)
• the top 3 frames from your own code, without line numbers

Library frames (node_modules, site-packages, the JVM and so on) are skipped. So the same bug on different hosts, for different orders, groups into one issue.

• Statuses: unresolved, resolved or ignored. If a resolved issue happens again, it reopens automatically and is marked regressed.
• Custom grouping: send fingerprint: ["checkout", "{{ default }}"] to extend the default fingerprint, or ["my-group"] to replace it.
• Explain: if the server sets ANTHROPIC_API_KEY, the issue page shows ✨ Explain, which sends the issue's export to Claude (claude-opus-5-5, with server-side refusal fallback on) for a likely cause and fix. It is cached per issue. Issue data leaves your server when you use it, and the dashboard asks once before sending.

#Log patterns

Oculo turns each message into a template by replacing values with <*>: numbers, ids, UUIDs, IPs, URLs, quoted strings, path segments containing digits, and the values in key=value pairs. So Charging order 8761 and Charging order 9283 become one pattern, Charging order <*>.

On the Logs page, switch to Patterns. It respects your current query and time range. Click a pattern to see its logs (pattern:<id>), or exclude it (-pattern:<id>). Patterns are capped at 5,000 per project.

#Metrics

Charts computed from your logs, with no separate metrics pipeline:

• Functions: count, avg, sum, min, max, p50, p90, p95, p99.
• Field: any numeric attribute, such as @duration_ms. Not needed for count.
• Grouped by service, environment, release, host, logger, level, or any attribute such as @route. You get the top 8 groups, and the rest fold into "other".
• Filter with any query.

Presets include "p95 latency by route" and "errors by service". You can click a series in the legend to hide it, and the table under the chart links to each group's logs.

API: GET /api/v1/metrics?fn=p95&field=@duration_ms&group_by=@route&q=service:api&since=24h. Percentiles scan at most the 500k most recent rows, and the response says when it was truncated. Use GET /api/v1/fields to discover attribute names and types.

#Alerts and heartbeats

When an alert fires:

• Threshold: at least N logs matching a query within M minutes.
• Spike: at least N matching logs and 3× the average of the previous 12 windows.
• New issue: a new issue appears.
• Regression: a resolved issue happens again.
• Heartbeat missed: a heartbeat's check-in is overdue (period + grace). A "recovered" notice follows the next check-in.
• Daily digest: a summary of the last 24h at a chosen UTC hour.

Where it's sent:

• Slack and Discord: an incoming-webhook URL.
• Webhook (JSON): any URL that accepts JSON. The payload includes alert, project, subject, message, details, url and fired_at.
• Email: SMTP, configured on the server.
• Telegram: a bot token plus a chat id.
• PagerDuty: an Events v2 integration key. Missed heartbeats resolve the incident on recovery.

A cooldown stops repeats, Send test checks the wiring, and History shows every delivery and its result. Secrets (bot tokens, routing keys) are masked whenever an alert is read back.

Heartbeats live on the Alerts page:

1. Create one with a slug, an expected period and a grace time.
2. Copy its curl command into the end of your job.
3. Add a Heartbeat missed alert.

#Releases and source maps

Set release in your SDK. The Releases page shows the error rate per release and compares two releases. Deploy markers appear on every chart.

For minified JavaScript, upload source maps per release. Stack traces are then rewritten to the original file, line and function before grouping. The original minified stack is kept, and the dashboard can show it.

oculo sourcemaps upload --release 1.4.2 ./dist          # uploads every *.js.map under ./dist

await Oculo.uploadSourceMap({ url: 'https://oculo.funonymous.in', key: 'oc_read_…(read/write)', release: '1.4.2', file: 'app.3f9a1c.js', map });

Uploading needs a read/write key or a member account; ingest keys can't upload. The release must match what the app reports.

#Using Oculo from an LLM

• MCP server (Claude Code, Claude Desktop, Cursor, …):
  Copy

  claude mcp add oculo -- npx -y @gouravsrf/oculo-mcp --url https://oculo.funonymous.in --key oc_read_…

  For Claude Desktop, Cursor and other clients, use the same command in their MCP config: "command": "npx", "args": ["-y", "@gouravsrf/oculo-mcp", "--url", "…", "--key", "oc_read_…"]. Give it a read key so the LLM can't change anything. The tools:

  • project_summary: the best first call
  • search_logs, get_log_context
  • list_issues, get_issue, explain_issue
  • compare_releases, list_patterns, query_metrics, list_fields, list_projects
  • resolve_issue: only with --allow-write and a read/write key
• REST for agents: every read endpoint takes:
  • format=json|md|text: md gives compact tables that fit more in context.
  • fields=a,b.c: trims each row.
  • limit, plus a next_cursor for paging. There are hard caps.
• Discovery: /llms.txt, /openapi.json and this page (/docs.md).
• Copy for LLM: a button on every log and issue copies a compact Markdown export you can paste into any chat.

A good agent workflow:

1. Call project_summary.
2. Call list_issues sort=freq.
3. Call get_issue on the top issue.
4. Use search_logs with issue:<id> or trace:<id> to see what happened around it.
5. Use get_log_context on one event.
6. Use compare_releases to check whether it started with a deploy.

#CLI

npm run build && npm install -g ./cli      # provides the `oculo` command (or: node cli/dist/index.js)
oculo login --url https://oculo.funonymous.in --key oc_read_…  # saved to ~/.config/oculo/config.json (mode 600)
oculo search 'level:>=warn service:api' --since 1h -n 50
oculo tail 'level:error'                    # live; reconnects automatically
oculo issues --sort freq
oculo summary
oculo sourcemaps upload --release 1.4.2 ./dist
oculo heartbeat nightly-backup

--json prints one JSON object per line, for piping into jq. Output is coloured on a terminal and plain when piped. OCULO_URL, OCULO_KEY and OCULO_PROJECT override the saved config.

#API reference

The base URL is https://oculo.funonymous.in. Errors look like {"error": "...", "hint": "..."}. Read endpoints accept format, fields and, for admin keys and users with several projects, project=<slug>.

The machine-readable version is /openapi.json.

#Configuration

Set these as environment variables on the server.

Per-project settings (in Settings, or PATCH /api/v1/projects/:id):

• retention days
• scrubEmails
• extraKeys (more sensitive key names to scrub)
• sampling (for example {"debug": 0.1})
• dropMessages (substrings to drop)

#Scaling: ClickHouse and Redis

Out of the box Oculo keeps everything in one SQLite file. That is fast and needs no setup, and it is the right choice for most teams: on a laptop it ingests about 20,000 events per second.

Two optional pieces take it further. Each one is turned on by a single environment variable, and they can be used separately or together:

Everything else (projects, keys, accounts, issues and their counters, rollups, patterns, alerts) always stays in SQLite. The dashboard, the query language, the API, the CLI and MCP behave the same in every mode; the test suite runs in all four combinations and a parity test checks that SQLite and ClickHouse return identical results for the same data and queries.

#When to use them

• Stay on SQLite while searches feel fast and the database file is a comfortable size. Most projects never need more.
• Add ClickHouse when you keep tens of millions of logs or more, when searches over long time ranges get slow, or when the SQLite file grows uncomfortably large.
• Add Redis when traffic is bursty (an outage that floods errors, a batch job), when you want ingest to keep answering quickly no matter what processing is doing, or when you run ClickHouse and want larger, fewer inserts.

For reference, on one laptop with 8 concurrent senders posting batches of 500 events, time until all 100,000 events were searchable:

These numbers measure ingest only. ClickHouse's main advantage, cheap storage and fast scans over very large tables, shows up at volumes far beyond this test.

#Turning on ClickHouse

1. Run ClickHouse (tested with 24.8). With Docker:
   Copy

   docker run -d --name oculo-clickhouse -p 127.0.0.1:8123:8123 \
     -e CLICKHOUSE_USER=oculo -e CLICKHOUSE_PASSWORD=change-me \
     -e CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 \
     -v oculo-clickhouse:/var/lib/clickhouse clickhouse/clickhouse-server:24.8

2. Start Oculo with:
   Copy

   CLICKHOUSE_URL=http://localhost:8123 CLICKHOUSE_USER=oculo CLICKHOUSE_PASSWORD=change-me npm run dev

   Oculo creates the database (CLICKHOUSE_DATABASE, default oculo) and the logs table on first start. The startup line says logs: clickhouse, and GET /healthz reports "storage": "clickhouse".

If ClickHouse can't be reached at startup, Oculo exits with a clear error instead of silently falling back, so a typo in the URL is obvious.

#How ClickHouse mode works

• Same ids, same API. Oculo allocates log ids itself (a sequence in SQLite), so cursors, live tail, "surrounding logs" and links from issues to their latest event work exactly as before.
• Counters and rows can't drift apart. When a batch arrives, issue counters, rollups and patterns are updated in one SQLite transaction, and the new rows are written to an outbox table in that same transaction. The outbox is then flushed to ClickHouse. Each flush carries a deduplication token derived from its log ids, so a retry after a crash never inserts rows twice.
• ClickHouse outages don't lose logs. If ClickHouse is down, ingest keeps succeeding; rows wait in the outbox and are inserted when it comes back (with backoff). GET /healthz shows outbox_rows while anything is waiting.
• Rows become searchable when they reach ClickHouse. Without the queue, the ingest request waits for the flush (normally milliseconds), so a log is searchable as soon as POST /api/v1/ingest returns.
• Search. Free text is matched word by word with the same rules as SQLite (word prefixes, phrases, exclusions): an n-gram skip index finds candidate rows and a word-boundary regex checks them. Attribute filters use ClickHouse's JSON functions with the same semantics (nested or dotted keys, numbers, booleans, missing values). Every value is a bound query parameter.
• Table layout. ORDER BY (project_id, ts, id), monthly partitions, bloom-filter indexes on trace, issue, user and fingerprint, and ZSTD compression for the search text.
• Retention and deletes use ClickHouse lightweight deletes: rows vanish from queries immediately and disk space is reclaimed by background merges.

#Existing data

Turning ClickHouse on does not move logs already in SQLite: new logs go to ClickHouse and old ones stay in the SQLite file (they are no longer shown). Issues, counts, timelines, rollups and patterns are kept, because they live in SQLite, and new log ids continue above the old ones. An issue whose latest event is still in SQLite shows no latest event until it happens again. There is no built-in tool to copy old rows into ClickHouse; keep the SQLite file until they would have aged out under retention anyway.

Use one ClickHouse database per Oculo installation (CLICKHOUSE_DATABASE). Pointing a brand-new SQLite database at a ClickHouse database that already holds another installation's logs would show those rows under the new installation's project ids.

#Turning on the Redis queue

1. Run Redis (tested with 7.4; 6.2 is the minimum for the stream commands used). Turn on persistence (AOF) so queued batches survive a Redis restart:
   Copy

   docker run -d --name oculo-redis -p 127.0.0.1:6379:6379 -v oculo-redis:/data redis:7.4 redis-server --appendonly yes

2. Start Oculo with REDIS_URL=redis://localhost:6379. The startup line says queue: redis.

#How the queue works

• Ingest answers right away. POST /api/v1/ingest checks the key, the size limits and the rate limit, appends the batch to the stream (OCULO_QUEUE_STREAM, default oculo:ingest) and returns 202 {"accepted": n, "dropped": 0, "queued": true}. Sampling and drop rules run later in the worker, so dropped is always 0 in queue mode.
• A worker drains it in batches. The server reads up to OCULO_QUEUE_BATCH messages (default 100) at a time through a consumer group and applies them in transactions of about 2,000 events, yielding between them so the HTTP side stays responsive (in tests, a 200,000-event backlog drained at ~40,000 events/s with no pause longer than a quarter of a second), then acknowledges them.
• Each message counts once. Message ids are recorded in the same transaction as the counters, so a message delivered twice (for example after a crash between commit and acknowledgement) is skipped.
• Crashed workers are covered. Messages a worker read but never acknowledged are reclaimed after OCULO_QUEUE_CLAIM_IDLE_MS (default 60 s).
• Bad messages don't block the queue. A message that fails three times while other messages are being applied is moved to <stream>:dead with the error; the rest of its batch is still applied.
• Outages don't empty the queue into the dead-letter stream. If every message fails (disk full, database locked), nothing is dead-lettered: messages stay pending, are retried when reclaimed, and back-pressure starts if the problem lasts.
• Back-pressure instead of unbounded memory. When more than OCULO_QUEUE_MAX_BACKLOG batches (default 50,000) are waiting, ingest answers 503 with Retry-After: 5. The SDKs and OpenTelemetry exporters retry automatically.
• Redis outages don't refuse logs. If Redis is unreachable while the server is running, ingest processes the batch in the request instead (and logs a warning once a minute).
• Acknowledged entries are trimmed from the stream every 30 seconds.
• GET /healthz reports queue_backlog, the number of batches not yet applied.

#Things to know about the queue

• Logs become searchable a few milliseconds after the request returns, not before. Scripts that write and then immediately read should poll, or check queue_backlog in /healthz.
• Oculo is still one server process: the queue gives burst absorption and durability, not multiple writers. The worker runs inside the server.
• One rare duplicate remains possible: if Redis accepts a batch but the connection drops before it answers, the server treats Redis as down, processes the batch directly, and the worker later applies the queued copy too.

#Both together

With CLICKHOUSE_URL and REDIS_URL set, a batch goes HTTP → Redis stream → worker (one SQLite transaction for counters + outbox) → ClickHouse. This is the highest-throughput setup and the one used by the Docker Compose file in Deployment.

#Settings

#Checking it

curl -s https://oculo.funonymous.in/healthz
# {"ok":true,"name":"Oculo","storage":"clickhouse","queue":"redis","outbox_rows":0,"queue_backlog":0}

In ClickHouse, SELECT count() FROM oculo.logs shows the stored rows. In Redis, XINFO GROUPS oculo:ingest shows the consumer group's lag and pending messages, and XLEN oculo:ingest:dead shows dead-lettered batches.

#Deployment

Oculo is one Node.js process. Pick the setup that matches your volume; you can move between them later.

#Node + SQLite

git clone <your fork of this repository> oculo && cd oculo
npm ci
npm run build -w server
PUBLIC_URL=https://logs.example.com OCULO_DB=/var/lib/oculo/oculo.db node server/dist/index.js

• On first start an admin key is printed and saved next to the database (admin-key.txt, mode 600). Open the dashboard and create the first account; it becomes the instance admin.
• Keep server/dist, server/public and docs/ together: the server serves the dashboard from server/public and the documentation from docs/.
• Run it under a supervisor so it restarts. A minimal systemd unit:
  Copy

  [Unit]
  Description=Oculo
  After=network.target
  
  [Service]
  WorkingDirectory=/opt/oculo
  Environment=PUBLIC_URL=https://logs.example.com
  Environment=OCULO_DB=/var/lib/oculo/oculo.db
  ExecStart=/usr/bin/node server/dist/index.js
  Restart=always
  User=oculo
  
  [Install]
  WantedBy=multi-user.target

• npm run dev (with tsx watch) is for development only.

#Docker (SQLite)

docker build -t oculo .
docker run -d --name oculo -p 4318:4318 -v oculo-data:/data -e PUBLIC_URL=https://oculo.funonymous.in oculo
docker exec oculo cat /data/admin-key.txt      # the bootstrap admin key

The image runs as the unprivileged node user, stores the database in the /data volume and has a health check on /healthz.

#Docker Compose: Oculo + ClickHouse + Redis

The repository's docker-compose.yml starts all three, with ClickHouse holding log rows and Redis Streams in front of ingest (see Scaling):

CLICKHOUSE_PASSWORD=pick-a-strong-password docker compose up -d --build
docker compose logs -f oculo        # look for: logs: clickhouse, queue: redis

Then open https://oculo.funonymous.in. Data lives in three named volumes (oculo-data, clickhouse-data, redis-data). ClickHouse and Redis are not published on host ports; only Oculo is.

#Behind a reverse proxy (HTTPS)

Terminate TLS in nginx, Caddy or a load balancer and forward to port 4318.

• Set PUBLIC_URL to the public https:// address. It is used in alert links, invite links, the GitHub OAuth redirect and the docs.
• Forward X-Forwarded-Proto so session cookies are marked Secure.
• Turn off response buffering for /api/v1/tail (Server-Sent Events). nginx: proxy_buffering off; (Oculo also sends X-Accel-Buffering: no).
• Allow request bodies up to MAX_BODY_BYTES (10 MB by default), e.g. nginx client_max_body_size 10m;.

Caddy example:

logs.example.com {
  reverse_proxy 127.0.0.1:4318 {
    flush_interval -1
  }
}

#Backups

• SQLite (always used for projects, keys, accounts, issues, alerts; and for logs unless ClickHouse is on): back up while running with sqlite3 /data/oculo.db ".backup '/backups/oculo-$(date +%F).db'". Copying the file alone while the server writes is not safe; copy the -wal file too or use .backup.
• ClickHouse: BACKUP DATABASE oculo TO Disk('backups', 'oculo-2026-11-15') (configure a backup disk), or snapshot the volume. Losing ClickHouse loses log rows but not issues, counts or settings.
• Redis: holds only batches not yet processed. With --appendonly yes they survive a restart; there is nothing to back up long-term.

#Upgrades

1. Back up the SQLite file.
2. Pull the new version, npm ci, npm run build -w server (or rebuild the image).
3. Restart. Database migrations run automatically on start (PRAGMA user_version); the ClickHouse table is created if missing.

With the Redis queue on, a restart doesn't lose ingest: batches wait in the stream and are applied when the server is back. Without it, SDKs buffer and retry while the server restarts.

#Sizing notes

• Retention is per project (default 14 days). Hourly rollups and issue timelines are kept for ROLLUP_RETENTION_DAYS (default 400).
• SQLite: plan disk for roughly the raw size of your logs plus the full-text index. Fast disks (SSD) matter more than CPU.
• If ingest is bursty, add Redis; if storage or long-range searches become the problem, add ClickHouse. See Scaling.

#Security and privacy

• Scrubbing at ingest: passwords, tokens, API keys, cookies, Authorization headers, JWTs and Luhn-valid card numbers are always scrubbed on the server. Masking emails is optional.
• Keys are stored as SHA-256 hashes and shown only once.
• Accounts: passwords use scrypt. Login is rate-limited and doesn't reveal whether an email exists.
• Sessions: HttpOnly, SameSite=Lax cookies (Secure behind HTTPS). They are never accepted on ingest. Cookie-authenticated writes must come from Oculo's own origin.
• GitHub sign-in verifies the OAuth state and links accounts only through a verified GitHub email.
• Live tail uses one-time 60-second tickets, so long-lived keys never appear in URLs.
• Agent keys: read keys can never write. Project keys can't create alerts, because alerts make outbound requests.
• Queries are fully parameterised, and the dashboard escapes all log content (HTML in logs is shown as text).
• Explain is off unless configured, and asks before sending issue data to Anthropic.

#Development and testing

#Layout

#Architecture

apps / SDKs / OTel ──► POST /api/v1/ingest ──► [Redis stream]* ──► worker
                                                                     │  one SQLite transaction:
                                                                     │  normalize, scrub, source maps,
                                                                     │  fingerprint → issues, rollups,
                                                                     │  patterns, log rows (SQLite)
                                                                     │  or outbox (ClickHouse)*
                                                                     ▼
                                                     SQLite (metadata, issues, …)  ──outbox──►  ClickHouse*
dashboard / CLI / MCP / agents ──► REST API ──► query compiler ──► SQLite FTS5 or ClickHouse SQL
                                                                                   * optional

• Server code (server/src):
  • ingest.ts: the pipeline
  • normalize.ts, scrub.ts, fingerprint.ts, patterns.ts, sourcemaps.ts: its steps
  • logstore.ts: SQLite and ClickHouse log stores
  • outbox.ts: ClickHouse outbox
  • queue.ts: Redis Streams
  • query/: the query language parser and compiler, with one compiler for both SQL dialects
  • api/: routes
  • alerts.ts, users.ts, migrations.ts
• Queries are written once with ? placeholders. For ClickHouse, numbers are inlined and strings become typed parameters ({p1:String}), so user input never reaches SQL text.

#Running locally

npm install
npm run dev            # server on https://oculo.funonymous.in with auto-reload
npm run seed           # (another terminal) ~24h of demo logs

#Tests

ClickHouse and Redis tests need running servers:

docker run -d --name oculo-clickhouse -p 127.0.0.1:8123:8123 -e CLICKHOUSE_USER=oculo -e CLICKHOUSE_PASSWORD=oculo-dev \
  -e CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 clickhouse/clickhouse-server:24.8
docker run -d --name oculo-redis -p 127.0.0.1:6379:6379 redis:7.4

export E2E_CLICKHOUSE_URL=http://localhost:8123 E2E_CLICKHOUSE_USER=oculo E2E_CLICKHOUSE_PASSWORD=oculo-dev
export E2E_REDIS_URL=redis://localhost:6379
npm run e2e:all && npm run test:parity && npm run test:resilience

Each run uses its own ClickHouse database and Redis stream and removes them afterwards.

External services (GitHub, SMTP, Telegram, PagerDuty, Anthropic) are always replaced by local stubs in tests; no test talks to a real third-party service.

#Docs

Each page in docs/ is plain Markdown. The server stitches them into one page in the order listed in server/src/docs.ts (DOC_ORDER), so a new page must be added there. Links between pages ([Metrics](#metrics)) become in-page anchors, and https://oculo.funonymous.in is replaced with the server's own address.

#Troubleshooting