Job Snapshots

Job snapshots give scheduled jobs a first-class, declared, validated place to record the numbers they compute — alongside ScheduledJob and JobExecution. Instead of burying metrics in log output or execution records, snapshots store already-aggregated figures that can be queried, charted, and rolled up over time.

This is deliberately not a general analytics store. Analytics implies high-volume event ingestion and aggregation. Snapshots are the opposite shape: a handful of already-aggregated numbers, once per run, per job.

Quick Start

1. Declare the snapshot shape on your handler

The snapshot declaration sits alongside the handler registration in your app code (see Scheduled Jobs for the handler registry pattern):

// YOUR APP CODE — handler registration with snapshot declaration
registerJobHandler('orders:fo-backlog-report', handler, {
  name: 'F&O Backlog Report',

  snapshot: {
    bucket: 'day',
    dimensions: [
      { key: 'segment', label: 'Segment', values: ['all', 'coupon'] },
      { key: 'group',   label: 'F&O Group',
        values: ['Backorder', 'Canceled', 'Delivered', 'Invoiced',
                 'in_fo', 'pending', 'total'] }
    ],
    metrics: [
      { key: 'qty',      label: 'Orders',       type: 'gauge', unit: 'count' },
      { key: 'avg_days', label: 'Average Days', type: 'gauge', unit: 'days',
        weightBy: 'qty' }
    ]
  }
});

2. Record snapshots in the handler

context.snapshot({ segment: 'all', group: 'Backorder' }, { qty: 781, avg_days: 22.95 });
context.snapshot({ segment: 'all', group: 'total'     }, { qty: 1586, avg_days: 28.43 });
context.snapshot({ segment: 'coupon', group: 'total'  }, { qty: 224, avg_days: 31.01 });

3. Query the series

import { JobSnapshotService, systemAuth } from '@generated/datalayer/services';

const svc = new JobSnapshotService(systemAuth());
const points = await svc.series({
  handler: 'orders:fo-backlog-report',
  dimensions: { segment: 'all', group: 'Backorder' },
  from: '2026-09-01',
  to: '2026-09-08'
});
// → [{ bucket: '2026-09-01', metrics: { qty: 781, avg_days: 22.95 }, ... }, ...]

Snapshot Declaration

The snapshot block sits beside configOptions in the handler registration, so a job's inputs and outputs are described in one place. Declaration is what makes validation possible.

`bucket` — Idempotency grain

Grain	Format	Example	Use case
`run`	Execution ID	`exec_abc123`	Every execution is its own data point
`hour`	`YYYY-MM-DDTHH`	`2026-09-08T14`	Hourly aggregates
`day`	`YYYY-MM-DD`	`2026-09-08`	Daily reports (most common)
`week`	`YYYY-Www`	`2026-W37`	Weekly summaries

The bucket string is derived from the grain and the job's configured timezone (falling back to UTC). A bucket: 'day' job in America/New_York produces 2026-09-08 at 11pm local time, not 2026-09-09.

Why bucketed upsert matters: Jobs run twice — a manual trigger plus the schedule, or an accident. Without bucketed upsert, that's two points for one day and a chart that lies.

`dimensions` — What the measurement describes

dimensions: [
  { key: 'segment', label: 'Segment', values: ['all', 'coupon'] },
  { key: 'group',   label: 'F&O Group' }   // free-form (no values list)
]

values is optional. If provided, the buffer validates calls against the declared values.
Omitted values makes the dimension free-form, subject to the cardinality cap (default: 1,000 distinct tuples per job).
All declared dimension keys must be provided on every context.snapshot() call.

`metrics` — What was measured

metrics: [
  { key: 'qty',      label: 'Orders',       type: 'gauge', unit: 'count' },
  { key: 'avg_days', label: 'Average Days', type: 'gauge', unit: 'days',
    weightBy: 'qty' }
]

Property	Required	Description
`key`	Yes	Metric identifier (used in `context.snapshot()` calls)
`label`	Yes	Human-readable name for UI display
`type`	Yes	`gauge` (point-in-time reading) or `counter` (cumulative)
`unit`	No	Unit label for display (e.g., `'count'`, `'days'`, `'ms'`)
`weightBy`	No	Companion metric key for weighted averages in rollups

Gauge vs counter: A backlog count is a point-in-time reading — averaging it over a week is meaningful, summing is nonsense. An error count is cumulative — summing is correct. Recording the type lets the rollup API refuse to sum a gauge.

Weighted averages: avg_days cannot be averaged across groups without weights. weightBy: 'qty' declares the companion metric so rollup() computes a correct weighted average. Without weightBy, rollup refuses to aggregate the metric rather than approximating.

Validation

The snapshot buffer validates every context.snapshot() call against the declaration:

Check	Error
Unknown metric key	`Unknown metric key: "typo". Declared: [qty, avg_days]`
Unknown dimension key	`Unknown dimension key: "segmnt". Declared: [segment, group]`
Dimension value not in declared `values`	`Dimension "segment" value "premium" not in declared values: [all, coupon]`
Missing required dimension	`Missing required dimension: "group"`
Non-finite number	`Metric "qty" must be a finite number, got: NaN`
Cardinality cap exceeded	`Cardinality cap reached (1000)`

Error handling: Errors throw in development (NODE_ENV !== 'production') and are demoted to logged warnings in production. A job that computes its numbers correctly should not fail because a label is wrong.

Storage

JobSnapshot Entity (Builtin)

Field	Type	Description
`jobId`	objectId → ScheduledJob	Which job produced this
`handler`	string	Handler key (survives job deletion/rename)
`jobExecutionId`	objectId → JobExecution	Traceability to the specific run
`bucket`	string	Idempotency key (ISO date, hour, week, or execution ID)
`bucketGrain`	enum	`run`, `hour`, `day`, `week`
`dimensions`	any	`{ segment: 'all', group: 'Backorder' }`
`dimensionHash`	string	SHA-256 prefix of sorted-key JSON dimensions
`metrics`	any	`{ qty: 781, avg_days: 22.95 }`
`capturedAt`	isoDate	When the snapshot was recorded

Unique index: (jobId, bucket, dimensionHash) — ensures bucketed upsert for idempotency.

Why dimensionHash? MongoDB's unique index on embedded objects is byte-order-sensitive — { a: 1, b: 2 } and { b: 2, a: 1 } are different keys. The buffer sorts dimension keys before hashing, making the index reliable regardless of how the caller constructs the object.

Row per dimension tuple, not per measurement. Co-computed metrics (qty and avg_days) stay together in one row. This is both fewer rows and more useful — 14/day for the worked example, ~5k/year.

Buffering and Partial Runs

context.snapshot() calls are buffered in memory and flushed once on success. If the job fails, the buffer is discarded — a job that fails halfway never leaves half a snapshot.

On discard, a summary is logged: "Discarded 10 buffered snapshots due to job failure".

Within a single run, calling context.snapshot() twice with the same dimension tuple overwrites (last write wins). This is useful when a job progressively refines its numbers.

Scheduler Integration

SnapshotBuffer is a generated utility (packages/jobs/snapshot-buffer.ts). The scheduler that uses it is your app code — Radish does not generate a scheduler (see Scheduled Jobs: What You Build).

Using SnapshotBuffer Standalone

SnapshotBuffer works independently of any particular scheduler implementation. You can use it anywhere you need validated, buffered metric recording with flush-on-success semantics:

import { SnapshotBuffer } from '@generated/datalayer/jobs/snapshot-buffer';
import { JobSnapshotService, systemAuth } from '@generated/datalayer/services';

const buffer = new SnapshotBuffer(snapshotDeclaration, {
  jobId: 'job-123',
  handler: 'orders:fo-backlog-report',
  jobExecutionId: 'exec-456',
  timezone: 'America/New_York'
});

// Record points — validated against the declaration
buffer.record({ segment: 'all', group: 'Backorder' }, { qty: 781, avg_days: 22.95 });
buffer.record({ segment: 'all', group: 'total' },     { qty: 1586, avg_days: 28.43 });

// Flush on success — writes to JobSnapshot via service
const svc = new JobSnapshotService(systemAuth());
const count = await buffer.flush(doc => svc.upsert(doc));
console.log(`Persisted ${count} snapshot points`);

// Or discard on failure — nothing is written
const { discardedCount } = buffer.discard();

Recommended: Integrating with Your Scheduler

When building a job scheduler (see Scheduled Jobs), wire SnapshotBuffer into the execution flow. This is the recommended pattern used by Radish production apps:

// YOUR APP CODE — inside your scheduler's executeJob() function

import { SnapshotBuffer } from '@generated/datalayer/jobs/snapshot-buffer';
import { JobSnapshotService, systemAuth } from '@generated/datalayer/services';

const handlerConfig = getJobHandler(job.handler);
let snapshotBuffer: SnapshotBuffer | null = null;

if (handlerConfig?.config?.snapshot) {
  snapshotBuffer = new SnapshotBuffer(handlerConfig.config.snapshot, {
    jobId: String(job._id),
    handler: job.handler,
    jobExecutionId: String(execution._id),
    timezone: job.timezone
  });
  context.snapshot = snapshotBuffer.record.bind(snapshotBuffer);
}

// After handler completes successfully:
if (snapshotBuffer && snapshotBuffer.size > 0) {
  const svc = new JobSnapshotService(systemAuth());
  const count = await snapshotBuffer.flush(doc => svc.upsert(doc));
  context.log(`Persisted ${count} snapshot points`);
}

// On handler failure:
if (snapshotBuffer) {
  const { discardedCount } = snapshotBuffer.discard();
  if (discardedCount > 0) {
    context.log(`Discarded ${discardedCount} buffered snapshots due to job failure`);
  }
}

Reading Snapshots

`series()` — Time series query

Returns raw metric values per bucket, suitable for charting.

const svc = new JobSnapshotService(systemAuth());

const points = await svc.series({
  handler: 'orders:fo-backlog-report',
  dimensions: { segment: 'all', group: 'Backorder' },
  from: '2026-09-01',
  to: '2026-09-08',
  sort: 'asc'     // default
});

Dimension filtering is partial-match: { segment: 'all' } returns all points where segment='all' regardless of the group value.

Param	Type	Default	Description
`handler`	string	—	Handler key (required)
`dimensions`	object	—	Partial dimension filter
`metrics`	string[]	all	Filter to specific metric keys
`from`	string	—	Start of bucket range (inclusive)
`to`	string	—	End of bucket range (inclusive)
`limit`	number	1000	Max results (capped at 10,000)
`sort`	`'asc'` \| `'desc'`	`'asc'`	Sort by bucket

`rollup()` — Aggregation

Aggregates across buckets using the metric declarations:

gauge → simple average (or weighted average when weightBy is declared)
counter → sum

const summary = await svc.rollup(
  {
    handler: 'orders:fo-backlog-report',
    dimensions: { segment: 'all', group: 'total' },
    from: '2026-09-01',
    to: '2026-09-08'
  },
  // Pass metric declarations for correct aggregation
  handlerConfig.snapshot.metrics
);
// → {
//     count: 8,
//     metrics: { qty: 1432.5, avg_days: 27.8 },
//     weightedBy: { avg_days: 'qty' },
//     from: '2026-09-01',
//     to: '2026-09-08'
//   }

Without metric declarations, all metrics are treated as gauges (averaged).

REST API

Standard CRUD is available at the generated endpoint:

# List snapshots for a handler
curl "http://localhost:5173/api/v1/jobSnapshots?handler=orders:fo-backlog-report&bucket=2026-09-08" \
  -H "Authorization: Bearer $TOKEN"

Design Decisions

Retention independence. JobExecution accumulates fast and will eventually be pruned. Snapshots are a separate entity with a separate retention policy. Default is keep-forever given the volume (~5k/year for 14 rows/day).

Cardinality cap. A job passing an order number as a dimension would generate unbounded rows. The buffer enforces 1,000 distinct tuples per job and refuses writes past it with a clear error.

No backfill. The series starts when the feature is adopted. Existing apps cannot reconstruct history from log output.

Handler key durability. Snapshots are keyed on handler (not job name or ID) so the series survives job renames and re-creation. Renaming the handler string itself would split the series.

Worked Example

The F&O backlog report emits 14 rows/day (2 segments × 7 groups). "Are coupon orders clearing faster or slower than the book?" becomes one series() query over {segment: 'coupon', group: 'in_fo'} vs {segment: 'all', group: 'in_fo'} — no parsing, no shape-guessing, no dependence on log retention.

Services — Service layer architecture
Settings & Encryption — Reading settings from internal code

Quick Start​

1. Declare the snapshot shape on your handler​

2. Record snapshots in the handler​

3. Query the series​

Snapshot Declaration​

bucket — Idempotency grain​

dimensions — What the measurement describes​

metrics — What was measured​

Validation​

Storage​

JobSnapshot Entity (Builtin)​

Buffering and Partial Runs​

Scheduler Integration​

Using SnapshotBuffer Standalone​

Recommended: Integrating with Your Scheduler​

Reading Snapshots​

series() — Time series query​

rollup() — Aggregation​

REST API​

Design Decisions​

Worked Example​

Related Documentation​