Group into Spans

A span is a run of consecutive events: everything in one five-minute window, every 1000 events, one burst of activity between quiet periods. Kelora reports on each span when it closes — when the first event past its end arrives, or when the input ends — so this works on endless streams as well as files. On a quiet stream, that report can come late.

kelora examples/api_latency_incident.jsonl --span 10m --freq status --span-summary

2025-01-20T14:00:00Z  events=200  status.200=200
2025-01-20T14:10:00Z  events=200  status.200=198 status.503=2
2025-01-20T14:20:00Z  events=200  status.200=184 status.503=16
2025-01-20T14:30:00Z  events=200  status.200=200

Choose how events are grouped

Option	A span closes	Typical use
--span 5m	at each aligned time boundary (:00, :05, …)	rates per minute or hour
--span 1000	after every 1000 events that passed the filters	batches
--span FIELD	when the value of FIELD changes	runs of the same request, job, or host
--span-idle 5m	after 5 minutes without events	sessions, bursts

Time spans and idle spans need timestamps (Work with Time). Only events that pass your filters are counted in a span, so -l error --span 5m --span-summary gives errors per five minutes.

Durations take ms, s, m, h, d, and w, alone or combined (1h30m). Windows have a fixed length and are aligned to the Unix epoch in UTC: --span 1d is a UTC day, and --span 1w runs Thursday to Wednesday, because 1970-01-01 was a Thursday. Months and years are not calendar months and years: M is 30.44 days and y is 365.25 days, so kelora warns when you use them.

--span FIELD closes whenever the value changes, so interleaved values (a, b, a) produce several spans for a. Sort the input first, or use --freq FIELD if you only need totals.

One row per span: --span-summary

--span-summary prints one row per span with its label, the event count, and the per-span value of every metric. Add --freq, --describe, or track_*() calls for the columns you need:

kelora examples/worker_bursts.jsonl --span-idle 5m --freq job --span-summary

2024-05-06T08:00:02Z  events=4  job.resize=3 job.thumbnail=1
2024-05-06T08:12:40Z  events=3  job.thumbnail=1 job.transcode=2
2024-05-06T08:41:10Z  events=5  job.resize=4 job.thumbnail=1

Format	Use for
--span-summary	text in a terminal, TSV when piped
--span-summary=tsv	long-format rows: label, metric, key, value (key empty for plain counts such as events) — for DuckDB, pandas, gnuplot
--span-summary=json	one JSON object per span, nested metrics intact; start/end for time and idle spans, first_ts/last_ts (first and last event) in every mode

A window that received no input produces no row, so a time series from --span-summary has gaps where nothing was logged — fill them downstream if you need a dense series. A window whose events were all removed by --filter or -l still gets a row, with events=0; windows outside --since/--until don't appear at all.

Counts, sums, averages, distinct values, minimums, and maximums are tracked per span. Percentiles, cardinality, and top-N are tracked for the whole run only; Kelora says so and leaves them out of the rows:

kelora examples/worker_bursts.jsonl --span-idle 5m --describe ms --span-summary

🔸 --span-summary omits 3 non-additive metric(s) (ms_p50, ms_p95, ms_p99): percentiles, cardinality and rankings have no per-window value, so each row carries only count, sum, avg, unique, bucket, min and max. Add -m for the cumulative table, or --span-close with a span.events loop for per-window percentiles.
2024-05-06T08:00:02Z  events=4  ms_avg=111.25  ms_count=4  ms_max=142  ms_min=48  ms_sum=445
2024-05-06T08:12:40Z  events=3  ms_avg=1550.33  ms_count=3  ms_max=2390  ms_min=51  ms_sum=4651
2024-05-06T08:41:10Z  events=5  ms_avg=106  ms_count=5  ms_max=125  ms_min=47  ms_sum=530

For per-span percentiles, use a --span-close script with span.events.

Your own report per span: --span-close

--span-close runs a script each time a span closes. Inside it, span describes the span that just closed:

kelora examples/api_latency_incident.jsonl -q --span 10m \
  -e 'track_inc("requests"); if e.status >= 500 { track_inc("errors") }' \
  --span-close 'let r = span.metric("requests"); let x = span.metric("errors");
                print(`${span.label}  ${x}/${r} failed  ${x * 100.0 / r}%`)'

2025-01-20T14:00:00Z  0/200 failed  0.0%
2025-01-20T14:10:00Z  2/200 failed  1.0%
2025-01-20T14:20:00Z  16/200 failed  8.0%
2025-01-20T14:30:00Z  0/200 failed  0.0%

Variable	Contains
span.label	the start time (time and idle spans), the field value (--span FIELD), or #0, #1, … (count spans)
span.start, span.end	window boundaries as datetimes; empty for count and field spans
span.first_ts, span.last_ts	timestamps of the span's first and last events, in every mode
span.size	number of events
span.metric("name")	this span's value of a metric, 0 if the span had none (() for a percentile, cardinality, or top-N, which have no per-span value, and for a minimum or maximum the span never recorded)
span.metrics	all per-span metric values as a map
span.events	the events themselves, as an array of maps
metrics	running totals since the start

-q suppresses the events, so only the hook's output remains. span.events is collected only when a --span-close script exists, and holds every event in the span in memory — fine for minutes, not for days.

Events and their span

In --exec, each event knows its span through meta.span_id, meta.span_start, and meta.span_end. With time spans, an event whose timestamp falls in a span that already closed is late (meta.span_status == "late"); it is passed through, but not counted in the closed span; Kelora warns with the number of events --span-summary and --span-close missed this way. Out-of-order input — merged files, parallel shippers — produces late events; sort it first if every event must count. meta.span_id itself is assigned per event, so tagging events and grouping them later stays correct on unsorted input.

Spans need sequential processing; with --parallel, Kelora warns and ignores it (details).