Sequencer

The Sequencer runs unattended imaging. You author a plan — targets, filters, exposures — and a set of policy phases that wrap the imaging run (open the roof, point, focus, then park and close up). The executor walks the plan on a background thread, drives the equipment through the Console, and publishes a live journal you can watch in the UI or read at /api/sequencer/state.

This page describes the executor in core/sequencer/run.py and the plan model in plan.py / structured.py. For the on-disk plan format see the Plans topic; for autofocus internals see Autofocus.

What a run is made of

A compiled plan is a flat list of timeline steps the executor walks in order. Wrapped around that list are up to seven policy phases — ordered action lists that run at lifecycle boundaries (start, error, suspend, resume, end). Plan-wide policy (constraints, focus triggers, cooling, meridian, guiding) rides along in a hidden notes step and is consulted between steps.

So a run looks like:

1. Guiding precondition (if the plan guides) — a stand-alone run refuses to start unless PHD2 is open and connected, with a clear message telling you to connect the guide camera + mount first.
2. Cooling on-start (if asked) — set the setpoint, optionally wait to settle.
3. on_start phase — gates (wait-for-safe, wait-for-darkness, wait-until-time, altitude), then actions (open roof, unpark, precise point, autofocus...).
4. The timeline — slew / filter / expose / wait / focus / guiding, step by step, with constraint, focus-trigger and meridian checks between exposures, plus optional per-frame quality acceptance.
5. Cooling on-end (warm up, if asked).
6. on_end phase — park, close roof, warm camera...

A pause drops into the on_suspend phase and holds; a resume runs on_resume and continues. A failed step runs the on_error phase and ends the run. A stand-alone run also drops a checkpoint at every step so an interrupted night can be resumed.

Timeline steps

Each step has a kind. The full vocabulary (validated in plan.py):

• slew — move the mount. Either a named target (to = "M42", resolved via the planetarium send/fetch) or explicit ra_h / dec_deg. After the blind slew the executor applies a settle dwell ([mount].settle_after_slew_s). With precise = true it then runs a plate-solve / sync / re-slew pass to tolerance.
• rotate — rotate to a sky position angle (pa_deg) via the plate-solve loop. Emitted per target when you set a PA.
• filter — select a filter by name (looked up in the wheel's reported names, case-insensitive) or by position index. An optional dwell ([camera].additional_safe_time_after_filter_s) follows the change.
• expose — take count frames of exposure_s seconds. Carries binning (omit to use [camera].default_binning), gain, offset, light (true for lights, false for darks/bias), object_name, filter_name. Each frame is one Console capture job, awaited before the next. Auto-dither (if the plan's guiding policy enables it) fires between subs.
• wait — sleep seconds (interruptible; pause doesn't count against it).
• wait_until — hold until a condition. Modes, first match wins: safe = true (SafetySupervisor reports not-actionable-unsafe), alt_min_deg + ra_h/dec_deg (target rises above an altitude — needs [site]), time_local = "HH:MM" (next local wall-clock occurrence), or time / when (absolute ISO8601). timeout_min bounds the wait; on timeout the step FAILS so its on_error decides.
• focus — move the focuser to position (optionally relative). NOTE: a focus step with autofocus = true returns "autofocus not yet implemented" — autofocus during a run is driven by the Focus policy triggers below, not by this step.
• start_guiding / stop_guiding — start PHD2 guiding + wait for settle / stop it. No-ops unless the plan's guiding policy is enabled and PHD2 control is allowed.
• dither — explicit dither request. Commands PHD2 when guiding+dither are enabled and control is allowed; otherwise records the request and no-ops honestly.
• track — tracking on/off (on = true|false).
• park / unpark — mount park state.
• flat_cover — open or close a CoverCalibrator flat panel (action = "open"|"close").
• warm_camera — run the cooler warm-up ramp (used at end-of-plan without a full on_end phase).
• run_script — run an external command at any body point via the sandboxed script executor (command — the script path plus inline args — with optional args, timeout_s, cwd, shell). Honours the [scripts] policy and the Stop flag, so a Stop aborts a long-running script promptly. Off unless you've opted into external scripts.
• set — assign a variable into the scope. notes — comment / no-op (also the carrier for plan policy).
• repeat — repeat a body N times (count must be a literal at compile time). loop_targets — iterate a targets list, binding $target_name, $target_ra_h, $target_dec_deg per iteration.

Steps are produced for you by the structured editor; you can also hand-write them in the TOML tab.

Policy phases

A phase is an ordered list of actions plus optional gates. The phase action catalog is the single source of truth, and the editor builds its menus from it. Each action has a fixed domain:

• obs (observatory): open_roof, close_roof, park, unpark, find_home, power_on / power_off (need a channel switch id), connect_power / disconnect_power, slave_dome / unslave_dome, dome_park, dome_find_home, dome_goto (azimuth), open_flat_cover / close_flat_cover.
• img (imaging): precise_point (slew + solve + center), slew_radec (ra_hours, dec_deg), autofocus, blind_solve (solve the current field and sync the pointing model), blind_resync (whole-sky hint-free solve + sync — recovers a mount that's lost where it's pointing), cool_camera (setpoint_c, optional ramp_minutes for a gentle ramp, wait_settle), warm_camera, connect_all / disconnect_all (bring every configured device up or down), start_guiding, stop_guiding, abort_exposure, run_script (command, timeout_s), notify (message, severity info|warn|critical — journals AND sends via the configured alerts channel, best-effort; safe in any phase).

The domain split is what makes a sequence safe to run inside a blockscript. When a run is injected (blockscript-supervised), every obs action is SKIPPED — the blockscript owns the roof/mount/power — and only img actions run. Stand-alone runs execute every action. Unknown action types are dropped on normalise, so a hand-edited plan can't inject an unsupported action.

The phases:

• on_start — gates then setup. A failure of a gating or imaging action here returns an error and ABORTS before any capture (then on_error runs to leave the rig safe). precise_point / blind_solve here point at the plan's first coordinate-bearing target (on a resumed run, at the resume target's field).
• on_error — runs when a timeline step aborts the sequence: park, close roof, warm camera, leave it safe.
• on_suspend — runs the moment a pause drops the run into PAUSED (stop guiding, abort exposure, park...). Author it in the editor's On Suspend tab like any other phase.
• on_resume — runs on the way back to RUNNING after a pause (reopen, unpark, re-cool, blind solve, restart guiding).
• on_end — park, close roof, warm camera, etc. Runs whether the sequence finished naturally OR was stopped, so the rig is always left safe.

(The structured plan normaliser defines on_start, on_error, on_suspend, on_resume, on_end. Legacy on_end bool flags — park / warm_up_camera — are synthesised into an on_end action list when no new-style list exists.)

On-start / on-resume gates

Before a phase's actions run, its optional gates hold until preconditions are met (used mostly on on_start and on_resume):

• wait_for_safe — hold until the Safety supervisor reports SAFE (a WARNING/UNSAFE/UNKNOWN verdict keeps waiting; no supervisor configured = don't block).
• wait_safe_dark — hold until the Sun drops below the darkness threshold (dark_sun_alt_deg, default −18° astronomical), optionally plus an extra offset (dark_offset_minutes). Needs your [site] coordinates; skipped if the site is unset.
• wait_until_utc — hold until an absolute ISO8601 time.
• altitude_min_deg — advisory at the phase level; the per-target altitude constraint still enforces it during the run.
• max_wait_minutes — caps the total gate wait; exceeding it fails the gate (which aborts on_start).

Per-action completion and wait

Each action is issued, then waited on until it actually completes before the next runs — never fire-and-forget, never parallel. Completion is confirmed three ways (from _WAIT_SPEC):

• status — poll the device's reported end-state: open_roof/close_roof (shutter open/closed, 60 s timeout), park/unpark (AtPark, 120 / 30 s), find_home (180 s), cool_camera (cooler settled, 300 s). The command methods return on command-accepted, so polling is what proves the action finished.
• job — poll a background job to succeeded/failed, then settle: precise_point, slew_radec, autofocus, blind_solve, blind_resync (300 s, 5 s settle for slews).
• dwell — no reliable end signal, so wait a fixed time: warm_camera, power_on/power_off, connect_power/disconnect_power, abort_exposure, guiding, connect_all/disconnect_all, notify.

Override per action via its params: wait_s (dwell / extra settle), settle_s, timeout_s.

Per-action on-fail policy

When an action fails or times out, its optional on_fail policy decides (_apply_on_fail):

• mode = "cancel" (default) — cancel the phase. In on_start this aborts the run; elsewhere it propagates.
• mode = "pass" — log it and move to the next action.
• mode = "retry" with retries (1–99) — retry up to N times, resuming from retry_from ("self", "phase_start", or an earlier action index). When retries are exhausted, fall through to then: cancel / pass / go_to_end.
• then = "go_to_end" abandons the remaining timeline steps and falls through to the on_end phase (the run still ends as FAILED).

With no policy, the legacy resolver runs: standalone runs default to cancel (a human Try-Again / Cancel / Let-it-pass prompt is a Stage-2 hook, not yet wired); injected runs always cancel so the blockscript's error handling takes over. An exec-count safety cap guarantees the retry loop terminates.

Plan-wide policy (between-step checks)

These ride in the hidden notes policy step and are consulted by the run loop, mostly before each expose.

Constraints

Altitude/hour-angle/time gates, read from the mount's live properties:

• altitude_min_deg / altitude_max_deg — skip the target when outside the band.
• hour_angle_min_h / hour_angle_max_h — HA = sidereal_time − RA, wrapped to ±12 h.
• end_by_utc — absolute ISO8601 time; once passed, the whole sequence ends.

A tripped altitude/HA constraint skips the current target (all its steps until the next slew) and continues — it does not abort the run. If no mount reports properties, the check is skipped rather than blocking.

Focus

Refocus triggers, checked before an expose (_should_refocus):

• every_n_frames — refocus after N captured frames (counter resets per target).
• every_x_minutes — refocus on a wall-clock interval.
• delta_temp_c — refocus when the focuser temperature drifts this many degrees.
• on_filter_change — schedule a refocus on the next expose after a filter change.
• focuser_position — fallback position when autofocus isn't available.

When a trigger fires, _do_policy_focus runs: if [autofocus].enabled AND a calibration exists, it runs Refocus via _run_sequence_refocus — the validated, always-inward run_refocus from the Autofocus topic, reading the saved focus_recipe.json sidecar (step / half-range / points) for a tight sweep around the current position, fitting the V, and moving to the mathematical minimum. It runs synchronously and cancelably (a watchdog ties the sequence Stop flag to the focus run's cancel, so Stop / Halt-all actually abort an in-progress refocus) and restores full-frame / bin 1 afterwards so science subs don't inherit the AF binning. If autofocus is off or uncalibrated it moves to focuser_position; otherwise it journals a no-op. AF failures during a run are fail-soft — journaled, never abort the sequence.

The global [autofocus].refocus_ triggers (refocus_enabled + every_n_frames / every_n_minutes / temp_delta_c / on_filter_change) overlay under* the plan's focus block (plan keys win) and route through this same Refocus path. With [autofocus].filter_offsets mapped, a filter change moves the focuser by the per-filter chromatic delta instead of triggering a full refocus. After a successful refocus, an optional [autofocus].focus_final_offset_steps nudges the focuser a fixed number of steps off the measured best-focus (for a known camera/imaging-train offset); it's fail-soft — a failed nudge is journaled, never aborts the run.

Cooling

• enable_at_start — set the setpoint and turn the cooler on before the first capture.
• setpoint_c — override [cooling].default_setpoint_c.
• wait_for_settle (default true) — wait for the camera to reach setpoint. If true and cooling fails, the run is ABORTED; if false, the run proceeds without waiting.
• warm_up_at_end — run the cooler warm-up ramp at the end (best-effort, never changes the outcome).

Cooling does nothing if [cooling].temp_control_supported is false.

Meridian

For German equatorial mounts. The plan's mode is resolved against [mount].flip_mode:

• inherit (the editor default; the same as no meridian block, and as the legacy ignore) - the mount setting decides.
• off - never act at the meridian in this plan, even when the mount is armed.
• halt / flip - this plan's choice OVERRIDES the mount setting. [mount].flip_mode = "off" is not a master switch: a plan set to flip will flip. When a plan widens the mount setting (plan flip over mount off) the run logs a WARNING, journals one meridian.override event, and the run monitor shows a banner for the whole run. Set the plan to inherit if the mount should decide. With inherit the clamps below come from [mount].
• max_minutes_past — only act once the target is this many minutes past the meridian.
• pause_before_min — hold exposures for this window BEFORE the meridian so a sub can't straddle the flip, then wait for the target to cross and reach the action point.
• recenter_after_flip — plate-solve / sync / re-slew after the flip.

Checked before each expose. halt ends the sequence cleanly when past the meridian. flip re-slews to the same RA/Dec to force the mount to swap pier sides, settles ([mount].flip_settle_s), and — when the driver reports side-of-pier — VERIFIES the side actually changed; if it didn't, it errors and stops rather than imaging on the wrong side. It refuses to flip if the mount won't report RA/Dec (it never defaults a missing Dec to 0). A flip won't refire within 5 minutes of the last one.

Guiding

The guiding policy (exposure_s, binning, dither cadence, settle gate) round-trips through the plan. When the plan enables guiding AND PHD2 control is allowed (phd2.allow_control):

• Each target auto-starts guiding once centred (waits for settle) and stops it after that target's exposures, so the next slew isn't fighting the guider.
• Auto-dither fires between subs on the dither_every cadence (dither_px, optional dither_ra_only), using the plan's settle gate (settle_px, settle_time_s, settle_timeout_s). The cadence counter is sequence-wide and resets per target, so it works for single-sub events and interleaved LRGB rotation alike and never wastes a dither after the last sub.
• Guiding enforcement — a plan that requires guiding will NOT start a sub while PHD2 reports a non-Guiding state (looping / stopped / star-lost). It holds until PHD2 recovers, then (on a 120 s timeout) fails the exposure step so its on_error decides (skip the target / abort) — it won't quietly shoot unguided subs.

If PHD2 control is off, guiding start/stop/dither become honest no-ops (they don't silently claim success). Set guide exposure/binning inside PHD2 directly if you don't drive it from the plan.

An optional guiding watchdog ([guiding_watchdog], default off) layers quality gates on top while guiding: it holds the next sub when the guide star is lost or RMS exceeds rms_max_px, waiting up to recover_timeout_s (polled every poll_interval_s) before failing the step.

Per-frame quality acceptance

An optional grader ([frame_acceptance], default off) inspects each light sub the moment it's saved and decides whether to keep it. Bias/dark/flat frames and non-light captures are always kept.

Each configured gate rejects a frame that fails it (set to 0 to disable that gate):

• min_star_count — too few detected stars (clouds, dew, a bump).
• max_hfr_px — soft focus / seeing blow-up.
• max_eccentricity — elongated stars (trailing, poor guiding), 0 = round … 1 = a line.
• max_background_adu — sky too bright (moon, light, an accidental short warm-up).

The grade is measured on a centred crop (central_region_pct, near the optical axis, for speed). A per_filter_json map lets you loosen the gates per filter — narrowband frames have far fewer stars and larger HFR than broadband, so a single global min_star_count would be wrong for them; any gate not overridden falls back to the global value.

On a reject:

• The frame is journaled as rejected (with the reason) and, when move_rejects is on, moved into a sibling rejected/ folder so it leaves your light stack.
• With reshoot_max > 0, the sub is re-shot up to N times (a rejected re-shoot doesn't count toward the plan's frame count).
• With abort_after_consecutive > 0, N rejects in a row fail the exposure step (the sky has gone) so its on_error decides.

Grading fails open: any error while grading keeps the frame, so a glitch never silently drops data. The running tallies (accepted / rejected / re-shot) appear in the sequencer status.

Run states and controls

idle → running → succeeded (happy path). Other terminals: failed (a step aborted, or on_start failed, or a go-to-end policy fired), stopped (Stop). running ↔ paused for pause/resume; stopping is the transient after Stop.

• Pause — the current step finishes first (an in-flight exposure completes — photons aren't wasted), then the run enters PAUSED, runs on_suspend, and blocks until resume. Resume runs on_resume and continues from the next step.
• Stop — requests abort: sets the stop flag, unblocks a pause, and calls Console.halt_all() to stand the equipment down (idempotent). In-flight captures/slews/AF are cancelled via their cancel hooks. on_end still runs so the rig is parked/warmed.
• Only one run at a time; start() raises if a run is already alive. injected = true marks a blockscript-supervised run (obs actions skipped). When a blockscript's own run_sequence action launches the plan and it SUCCEEDS, the sequencer signals body_done, moving the blockscript RUNNING → SHUTDOWN so the night packs up. A scheduler CHUNK runs injected too but does NOT signal body_done, so a many-chunk night isn't ended after the first chunk.

Checkpoint and resume

A stand-alone run drops a checkpoint at every step boundary (and per-sub inside an exposure), so if a night is cut short — a crash, a power loss, an app restart — you don't lose the whole session. The checkpoint is kept on a crash / FAILED / STOPPED run and dropped on a clean SUCCEEDED completion, so re-running a finished plan always starts fresh.

Resume is target-level: on resume, the on_start phase re-runs (re-establishing pointing / focus / guiding after the downtime), completed targets are skipped, and the run restarts at the in-progress target's slew so it re-acquires and re-shoots that field (resuming mid-target without re-slewing would image the wrong sky). Subs already captured in the resume target are skipped, so you don't re-shoot what you already have.

One checkpoint is kept per plan. A resume is refused if you've edited the plan since the interrupted run (the step list no longer matches) — start fresh instead. Supervised runs (blockscript / scheduler) don't checkpoint; the blockscript or scheduler owns night-level completion.

Per-step error handling

Every timeline step carries an on_error policy (plan.py):

• abort (default) — failure runs the on_error phase and FAILS the run.
• skip — log it and continue to the next step.
• retry with retry_count — retry up to N times (with a short backoff), then skip.

The structured compiler picks sensible defaults: expose defaults to retry × 2; filter changes, track, guiding start/stop, per-target start conditions and end-of-plan park all default to skip (a sticky wheel or a missed start window shouldn't kill the run); slews default to abort.

Journal and status

Every step and synthetic action appends a StepEvent (timestamp, seq, kind, label, state, message, detail) to the journal and publishes it on the bus (sequencer.event). state is one of started / ok / skipped / failed / retrying (steps) or paused. Top-level status (sequencer.state) carries the run id, plan name, state, current/total step, progress fraction, error, and the last 200 journal events. It also reports the live frame-acceptance tallies (accepted / rejected / re-shot) and, while a refocus runs, a live autofocus V-curve so the run view can draw the focus graph as it happens.

Gotchas

• The legacy in-app focus step with autofocus = true is not implemented — use the Focus policy triggers for autofocus during a run.
• repeat.count must be a literal number at compile time; variable references there are rejected.
• Altitude/HA constraints rely on the mount exposing live properties (altitude, sidereal_time, RA). No mount, no enforcement — the check is skipped, not failed.
• A meridian flip that can't confirm the pier side changed STOPS the run for safety rather than continuing blind.
• Guiding/dither only act when PHD2 control is explicitly enabled; otherwise start/stop/dither are honest no-ops. A guiding-enabled stand-alone run refuses to start if PHD2 isn't connected.
• Per-frame acceptance and the guiding watchdog are off by default; turn them on in config. Acceptance grades light frames only and fails open (keeps the frame) on any grading error.
• Resume is refused if the plan was edited after the interrupted run, and only stand-alone runs checkpoint (blockscript/scheduler runs don't).
• on_suspend is fully authorable from the editor's On Suspend tab (fixed in v20.89 — earlier builds silently dropped what you authored there; re-save any plan that relied on a hand-added TOML list).

How it ties in

• Console is the single command surface the executor drives (mount, camera, filter wheel, focuser, roof, power, plate-solve, autofocus) — see the Console topic.
• Plans documents the on-disk TOML format and the hidden policy step.
• Autofocus covers the 2-button Wizard / Refocus model and the focus_recipe.json sidecar that the focus triggers invoke (via Refocus).
• Blockscript runs can host a sequence as an injected run; the obs/img domain split keeps the two from fighting over the observatory.
• The Safety supervisor backs the wait_for_safe gate and the safe wait_until mode.