Skip to main content

Metrics and diagnostics

Use status to check what is running, metrics to follow changes over time, and live diagnostics to inspect Lua or plugin output.

QuestionWhere to look
Has my Tenon Document been applied?GET /pipelines/{id}: compare documentEtag and appliedDocumentEtag
Is a plugin failing or restarting?Pipeline status and the plugin's diagnostics
How is workload changing?GET /metrics, collected periodically
Did a message reach its destination?The destination's own subscriber, database, or service

Collect metrics​

The default response is Tenon's structured JSON:

curl --fail http://127.0.0.1:18080/metrics

For Prometheus text:

curl --fail 'http://127.0.0.1:18080/metrics?format=prometheus'

?format=json explicitly selects JSON; this is not OTLP JSON. Use the metric catalog for names, units, attributes, and temporality, and the HTTP API for query filters.

When reading metrics:

  • Treat absent data as missing, not zero. Busy, slow, or disconnected pipelines may miss the collection deadline.
  • Account for process restarts when calculating rates from cumulative values.
  • Calculate percentiles in your monitoring system; Tenon does not calculate them.
  • Use catalog names in filters, before any exporter-specific renaming.
  • Treat running as runtime state, not proof of broker connectivity or delivery.

Runner labels and collection timing are startup settings in the Runner Configuration Schema.

Follow live diagnostics​

Choose one target per request:

TargetOutput
plugin:<instanceId>One plugin process's stdout and stderr
flow:<flowId>/channel:<index>One channel's lua-print and lua-error records

For the console plugin in First Pipeline:

curl --fail -N -H 'Accept: text/event-stream' \
'http://127.0.0.1:18080/pipelines/debug/diagnostics?target=plugin%3Aconsole'

For the first channel of its probe Flow:

curl --fail -N -H 'Accept: text/event-stream' \
'http://127.0.0.1:18080/pipelines/debug/diagnostics?target=flow%3Aprobe%2Fchannel%3A0'

Targets are URL-encoded in these commands. The stream can attach before the selected object starts.

EventMeaning
attachedThe subscription is open
diagnosticA Lua or plugin output record
closed, or end of streamThis subscription has ended

Read diagnostic records​

  • Use process, channel, and VM identities to distinguish restarts.
  • Sequences are decimal strings. Gaps can indicate dropped records; plugin stdout and stderr share one process sequence.
  • For Lua errors, use stable phase and code fields. The error text is a human-readable hint.
  • Text is limited to 16,384 UTF-8 bytes per record. Truncation and invalid UTF-8 are reported.

Diagnostics are live and best effort. They have no history, replay, or Last-Event-ID resume. A slow reader may lose records. Reconnecting creates a new subscription and repeats HTTP authorization.

Keep the Runner's stderr as well: startup, storage, and cleanup failures may occur when the HTTP API is unavailable.