Pace — architecture (systems reference)

This is the canonical detailed architecture reference for Pace — the per-system bullet list that previously lived inline in the repository AGENTS.md. It was extracted to keep AGENTS.md a concise agent bootloader. The doctrine and the high-level constellation diagram live in overview.md; per-file responsibilities live in ../development/key-files.md.

App shape

App Type: Menu-bar-first (LSUIElement=true) with no Dock icon. On MacBooks with a camera housing, the always-available Living Notch signal and quick panel are joined by one on-demand resizable Command Center window. Displays without a reported housing use a conventional status item and standalone panel; Pace never draws a synthetic center notch.
iPad companion target: PacePad is a full-screen iPadOS 26 input/output surface in the same Xcode project. It owns Bonjour discovery, authenticated reconnect, tap-to-talk recording, local system TTS, the companion face, low-rate local presence detection, and requested-only camera still capture. It owns no planner, durable memory, Mac inspection, tool execution, or interruption policy.
Framework: SwiftUI (macOS native) with AppKit bridging for onboarding, the Command Center window, menu-bar panel, hardware-derived notch overlay, and cursor overlay
Pattern: MVVM with @StateObject / @Published state management
Concurrency: @MainActor isolation, async/await throughout
Analytics: local-only no-op/timing-safe hooks via PaceAnalytics.swift; no cloud analytics SDK is linked.

iPad companion transport

PaceCompanionServer is a thin Mac adapter over the existing conversation, local audio transcription, privacy-pinned vision, world observation, and proactivity pipelines. It advertises _pace-companion._tcp and never creates a second planner or memory store.
PaceShared defines versioned messages and a length-prefixed frame containing bounded JSON control data plus optional AAC/JPEG bytes. Heartbeats and a bounded message-ID window handle connection health and replayed frames; shared policy also validates semantic payloads, stale sessions, privacy gates, heartbeat expiry, and bounded reconnect timing.
Pairing uses a six-digit, in-memory code only to establish TLS 1.3 PSK transport. The Mac then issues a random 256-bit credential stored in each device’s Keychain. Reconnect PSKs and a session HMAC derive from that credential; no secret is stored in UserDefaults or logged.
The iPad sends ordinary utterance audio and semantic presence events. JPEG bytes are sent only in response to an expiring camera_frame_request created by the conservative explicit physical-scene request parser. Pause or camera disable discards queued presence, cancels pending still continuations, and is re-checked before either side sends or accepts camera media. Temporary Mac audio files are removed after local transcription and requested images are never persisted.
If the Mac’s selected planner crosses the on-device boundary, the existing audit/tint behavior remains authoritative and the iPad processing and speaking states become amber for the same turn.

Planner

Planner: main screen/action planning uses the local OpenAI-compatible LM Studio reasoner. The current source default for both planner and fallback VLM is one resident qwen/qwen3.5-4b multimodal model, eliminating the missing-model and model-swap stalls caused by the previous separate Gemma/UI-Venus defaults. Pure-knowledge turns use the bundled in-process MLX planner when enabled; otherwise they use LM Studio’s native /api/v1/chat stream with reasoning: off and fall back before first output to the OpenAI-compatible route with both supported non-thinking hints. Structured action turns stay on Chat Completions for JSON-schema decoding. <think>…</think> stripping remains defensive. The bundled path reuses a complete Hugging Face CLI cache and runs one real inference at launch; an early user turn awaits that same warmup task instead of racing a second model load. The 2026-08-27 qualification scored 16/19 on the legacy planner fixtures and 7/8 on production v10 action fixtures, with the focused five-turn set averaging 878 ms. Gemma 3 12B remains an optional quality-over-memory planner and qwen/qwen3-30b-a3b remains the stronger 48 GB swap. Apple Foundation Models is scoped to classification and structured extraction rather than world-knowledge answers. See ../development/info-plist-switches.md. No cloud LLM. First-launch tier default: on a brand-new install with no pace.planner.tier. UserDefaults state at all, PacePlannerTierStore.loadConfiguration() checks SystemLanguageModel.default.availability and resolves to .appleFoundationModels when Apple Intelligence is available, .local otherwise. Existing persisted tier choices remain authoritative.
Planner (BuddyPlannerClient): the active planner is chosen by the user in Settings → Planner via PacePlannerTier (Local / CLI bridge / CLI direct / Direct API / Apple FM). Default local model is qwen/qwen3.5-4b. BuddyPlannerClientFactory.makeDefault() dispatches on tier: LocalPlannerClient for .local and as a fallback for missing/invalid configs; AppleFoundationModelsPlannerClient for .appleFoundationModels; CloudBridgePlannerClient for .cliBridge; PaceLocalCLIPlannerClient for .cliDirect; and DirectAPIPlannerClient for .directAPI. The same CompanionSystemPrompt flows through every tier. Direct-API keys live in macOS Keychain via PaceKeychainStore. Any off-device tier tints the capsule amber and fails loud unless the user explicitly enables local fallback. The main action planner is decode-constrained to the v10 {spokenText,intent,payload} envelope; invalid structured output retries without prompt cache. Eval coverage lives in evals/fm-fixtures-actions/ and is scored on decoded actions.
- .cliDirect general brain (OpenSpec openspec/changes/archive/2026-07-12-codex-general-brain): a user-selectable tier that direct-spawns the user’s already-authenticated codex (default) or claude CLI — no Node bridge, only the CLI on PATH — as the brain for ALL turns (the same PaceLocalCLIPlannerClient the .research lane already uses). Upstream sub-selection persists via PacePlannerTierStore key pace.planner.tier.cliDirect.upstream (PaceLocalCLIUpstream, default .codex). Off-device, so it inherits the FULL cloud-bridge safety contract via a separate transport-aware consent: PaceCloudBridgeConsent.hasAcceptedDirectSpawnConsent() + a 24-hour direct-spawn soak (canRunDirectSpawnTurn(now:)) — accepting the Node-bridge consent does NOT auto-grant the direct-spawn path. A .cliDirect turn tints the capsule amber (isOffDeviceTurnInFlight, keyed on plannerClientForThisTurn is PaceLocalCLIPlannerClient), writes a PaceAPIAuditLog entry (subsystem: "planner.cliDirect", target: <codex|claude>) so the Privacy dashboard flips off “0 bytes → X KB to codex”, and FAILS LOUD via PaceFailureNarrator on CLI error (no silent local fallback beyond the existing opt-in). Missing-binary preflight (PaceLocalCLIPlannerClient.isUpstreamBinaryOnPath) surfaces a plain-language “needs codex on PATH” hint before the turn. Codex is the phone-a-friend default: the .research lane defaults to Codex on fresh installs (PaceResearchTierStore.defaultCLIBridgeUpstream = .codex, defaultCLIBridgeModel = "" so Codex uses its own authenticated model; persisted upstreams honored for existing users), and cron/scheduled fires (CompanionManager+Lifecycle executeTaskCallback) route through a Codex direct-spawn brain ONLY when direct-spawn consent + 24h soak are satisfied (BuddyPlannerClientFactory.cronTaskBrainDecision, sharing cliDirectDispatchDecision) — otherwise they fall back to the local planner, so an unattended background fire never silently goes off-device (amber forced for the Codex path). The planner brain is also selectable from Settings → Models via a RAM-aware brain picker folded into PaceModelMemoryBudget’s budget section (PaceBundledModelsSettingsTab.memoryBudgetSection): picking a cloud/Apple-FM brain frees the local-planner RAM and the section live-updates the per-model breakdown, fits verdict, and largest-VLM-that-fits recommendation. Both the Planner tab and the Models picker route through the shared CompanionManager.selectPlannerTierWithConsent.

Voice, vision, TTS

Speech-to-Text: Apple SFSpeechRecognizer (on-device, requiresOnDeviceRecognition=true) is the default active provider — instant, no model download. TranscriptionProvider=whisperKit is accepted as a scaffolded local provider selection, but falls back to Apple Speech until the real WhisperKit streaming bridge lands. All cloud STT providers have been removed.
Text-to-Speech: two on-device conformers of BuddyTTSClient. The dependable default is LocalTTSClient (AVSpeechSynthesizer, Premium > Enhanced > compact LocalTTSVoiceIdentifier) because it has no sidecar or model-readiness dependency. TTSProvider=localServer opts into LocalServerTTSClient, backed by a loopback Kokoro-82M /v1/audio/speech service with per-utterance Apple fallback and a 30-second outage memo. Cloud TTS has been removed.
Local Vision-Language Model (optional): LM Studio at http://127.0.0.1:1234/v1 (OpenAI-compatible). The default qwen/qwen3.5-4b model is shared with the local planner, so only one model stays resident. Accessibility and native OCR run first; the VLM is the fallback for screens those native paths cannot expose. Planner/VLM HTTP roots are loopback-only. Pure-Q&A typed turns do not prewarm or call the VLM.
Screen Capture: ScreenCaptureKit (macOS 14.2+), multi-monitor support
Voice Input: Push-to-talk via AVAudioEngine + pluggable transcription-provider layer. System-wide keyboard shortcut via listen-only CGEvent tap.
Voice Input UI: PaceNativeInterfaceModel maps ready, listening, understanding, approval, acting, speaking, completed, blocked, and failed into one semantic state language. PaceSignalView renders that language across onboarding, the real notch, and the quick panel; off-device planner activity overrides the active signal to amber. Timeline rendering exists only for active states, and Reduce Motion uses static or short crossfade presentations.
Cursor: Codex-style arrow (CodexArrowShape) with linear-gradient fill, white highlight stroke, dual shadow.
Element Pointing: the planner embeds [POINT:x,y:label:screenN] tags in its response. The overlay parses these, maps coordinates to the correct monitor, and animates the cursor along a bezier arc to the target.

Memory

Two-tier in-context memory: every planner call carries the last K=4 turns verbatim (PaceThreadMemory.verbatimWindow()) AND a rolling summary of everything older (injectionPrefix(), rendered into the system prompt as <conversation_so_far>...</conversation_so_far>). The summary is refreshed by a detached Apple FM call after each turn via PaceThreadSummarizer; the user-facing path never blocks on summarization. Race-safety lives in a monotonic summaryVersion snapshot captured before each detached call, so out-of-order arrivals are dropped at applySummaryUpdate. In-session a 20-minute idle threshold still drops the live state (or Settings → Memory → Reset thread). The conversation now SURVIVES quit/relaunch: PaceThreadMemory.snapshot(now:)/restore(from:) are pure value-type accessors and PaceThreadMemoryStore persists a single on-device JSON (~/Library/Application Support/Pace/thread-memory.json, atomic write) after every turn/summary/reset; CompanionManager.start() rehydrates it. Policy is “resume always, until reset” — no staleness expiry; the file is wiped only on an explicit thread reset or when thread memory is disabled. The summary is still never journaled to paceHistory (only session id + lifecycle events are). Durable facts go through episodic memory instead.
Always-On Companion Mode (default off): typed provenance-bearing PaceWorldObservation evidence uses bounded atomic persistence, current-state/last-seen/change/presence queries, confidence decay, correction supersession, deterministic memory promotion, and a dedicated retrieval source. PaceCompanionModeController exposes off/starting/observing/interpreting/paused/degraded/privacy-blocked; PaceCompanionInterventionPolicy defaults to silent memory and keeps card/speech opt-ins separate. Settings → Companion and menu-bar indicators expose source consent, active capture, retention/storage, readiness, pause, and clear. App lifecycle starts explicitly enabled ambient/watch sources plus a production low-rate AVFoundation/Vision camera client that emits only non-identifying ephemeral person tracks and conservatively matches user-taught, locally persisted Vision feature prints across coarse camera zones. Teaching is an explicit centered-object capture; no photo is persisted. Ambient voice uses a bundled local pre-STT Core ML gate (PaceWakeWordClassifier, exact labels hey_pace and background); missing or malformed assets fail closed before Speech.framework is reachable. Accepted wakes hand off to the bounded push-to-talk conversation path. Silent cards and spoken interventions are separately default-off and wired through the intervention policy; speech additionally passes the existing live restraint/cooldown path. Routine promotion requires at least three unique supporting observations. On 2026-07-13 the owner directed “push through,” accepting the remaining unmeasured live/hardware and manual Cmd+R risk without representing those checks or documented thresholds as passed. See docs/product/companion-mode-privacy.md, docs/product/companion-mode-dogfood.md, and docs/product/pace-wake-word-classifier.md.

Actions, tools, integrations

Action Layer (agent mode): the planner should prefer <tool_calls> JSON blocks where the outer array is sequential steps and each inner array is a parallel group. Tool metadata lives in PaceToolRegistry so prompt docs, aliases, and risk labels share one source of truth; the local registry validates at app startup so schema examples, aliases, and enum coverage cannot silently drift. Legacy tags are still accepted: [CLICK:x,y], [DOUBLE_CLICK:x,y], [TYPE:text], [KEY:cmd+s], [SCROLL:up:3], [OPEN_APP:Safari], [OPEN_URL:https://example.com], [MUSIC:play], [VOLUME:up:2], [BRIGHTNESS:down:3], [CALENDAR:today], and [REMINDER:send invoice]. PaceActionTagParser extracts them; CompanionManager asks for user approval only for higher-risk non-undoable or external actions when the Approve Risky Actions preference is on, and the approval alert defaults to Cancel. Routine local actions such as click, scroll, window snap, app/URL open, media, volume, brightness, clipboard read, and undo can execute without the popup; blocking preflight issues still surface before execution. PaceActionApproval keeps the allow/cancel decision testable; PaceActionExecutor posts events after TTS playback starts. Single-clicks try PaceAXTargeter first (AX-tree press), falling back to CGEvent. AX set-value edits are logged in a session-local mutation log and can be restored with Undo.last / “undo that”. App launch/URLs use NSWorkspace; volume, brightness, media, and clipboard reads use local macOS APIs; Calendar/Reminders use EventKit for reads and creation. Mail recipient names can resolve through Contacts before composing a draft. Finder/Notes/Mail/Things/Shortcuts/Messages have first-pass local integrations. download_file downloads a user-named http(s) URL into ~/Downloads with URL validation, filename sanitization, and Finder-style collision suffixes — always approval-gated. Gated by EnableActions=true in Info.plist. Runtime smoke hooks are disabled unless PACE_ENABLE_SMOKE_HOOKS=1.
Trust surfaces (visible undo + reply replay + failure narration): every reversible mutation (createNote, appendNote, createReminder, createCalendarEvent, composeMail, createThingsToDo, runShortcut, downloadFile, recordFlow/runFlow, setTextValue/editSelectedText, openMessages with body, and generic mcp) raises a 5-second floating PaceUndoBanner next to the cursor; tapping it submits Undo.last through the executor. After every assistant turn finishes speaking, the notch panel renders a 30-second reply-replay button driven by the same post-processed string that flowed through TTS — replay does NOT re-stream the planner. Plain-language failures (planner offline, missing permission, click missed, sidecar TTS offline, MCP server not configured, cloud-bridge upstream error) are composed deterministically by PaceFailureNarrator (no LLM call) and routed through PaceRestraintGate.decide(...) so failure speech stays silent during a Zoom/active call. Reversibility uses the same set as PaceActionApprovalPolicy.actionIsReversibleMutation returns true for, and the click-missed narrator is wired to the all-fail click observation in PaceActionExecutor.
Typed automation catalog: Versioned manifests under Resources/automations/ contain canonical Pace tool calls, provenance, execution mode, required preferences, invocation examples, and ordered step groups. The indexed 17-entry starter library covers Notes templates, Calendar reads, timers, Finder locations, current-window layouts, bounded media control, weekly review, and end-of-day reset. PaceAutomationDefinitionLibrary validates every manifest at startup against PaceToolRegistry, rejects input injection, network/download, destructive, nested-flow, and opaque Shortcut capabilities, and calls PaceAutomationCompiler to reject partial parses. The compiler feeds the existing PaceActionTagParser, so typed automations reuse normal preflight, approval, executor, observation, and audit behavior. PaceAutomationCatalog builds a non-persisted view over typed automations (deterministicLocal), bounded Pace Programs (deterministicProgram), recorded flows (deterministicReplay), skills (plannerGrounded), and installed Shortcuts (externalOpaque). Explicit list/run commands still resolve locally by unique exact normalized name. Ordinary completed text then uses exact authored invocation aliases or the local Nomic/Apple embedding chain; token overlap never authorizes execution. A unique semantic score above both the absolute threshold and winner-margin gate dispatches directly. Close candidates can be resolved by a fresh typed-output Apple Foundation Models session that runs fully on-device, may choose only from the embedding shortlist, validates an opaque candidate identifier, and can decline or ask for clarification. Unavailable models and weak or failed matches preserve the original planner fallthrough. Catalog metadata is never added to any off-device planner prompt or conversation memory.
Programmable teachable skills: PaceProgramDefinition is a versioned declarative graph stored under ~/Library/Application Support/Pace/programs/. It supports ordered typed-action steps, conditions over one pre-run snapshot (weekday, local hour, or frontmost bundle identifier), and literal repetition. The whole graph—including inactive branches—is validated before save, discovery, and each run. Fixed limits are nesting depth 4, 50 source nodes, repeat count 10, and 50 worst-case expanded action steps. Programs cannot contain Lua, JavaScript, shell, imports, network/file APIs, raw Accessibility operations, or branches on action output. The selected graph flattens into a transient typed definition and reuses PaceAutomationCompiler plus normal preflight, approval, executor, observation, and audit behavior; no model runs after catalog selection. “Teach a skill …” and “Create an automation …” share a privacy-pinned local authoring ladder: fixed typed automation first, bounded Pace Program second, planner-grounded .skill.md last. The local model is an authoring aid, never runtime authority.
Teachable skills: .skill.md skills (PaceSkillLoader) are the natural-language sibling of typed automations, Pace Programs, and recorded flows — a numbered step list the planner re-grounds each run (intent tier), rather than compiling canonical tool calls (typed/program tier) or replaying recorded AX steps verbatim (flows, pixel tier). Bundled skills live in Resources/skills/<slug>.skill.md; user skills in ~/Library/Application Support/Pace/skills/. Teachable by telling (PRD docs/product/prds/teachable-skills.md): explicit teach/create transcripts use the shared representation ladder above; Settings → Skills → “Teach a skill” remains an explicit planner-grounded prose form. skillStructuringSystemPrompt + skillFromStructuredJSON use the privacy-pinned LOCAL text planner, with structureSkillDeterministically as a no-model fallback. Stored triggers are active catalog invocation phrases. PaceSkillLoader.save/deleteUserSkill/listUserSkills use the atomic temp-file+rename pattern and reuse PaceFlowStore.slug(for:). PaceSkillsView surfaces prose skills and deterministic programs in “Your skills.” Fully on-device — authored reusable work never leaves the Mac.
MCP Integration Layer: broad external integrations should use OSS Model Context Protocol servers instead of being rebuilt inside Pace. PaceMCPClient loads stdio server definitions from ~/.config/pace/mcp-servers.json or ~/.pace/mcp-servers.json, accepts common mcpServers config shape, runs MCP initialize + tools/call, and returns observations through the same approval/result loop. Planner MCP calls use either {"tool":"mcp","server":"altic","name":"notes_create","arguments":{...}} or {"tool":"notes_search","server":"altic","query":"..."} for server-native names. Missing configured server names are surfaced by PaceToolPreflight before approval. The bridge speaks newline-delimited JSON-RPC and is validated end-to-end by PaceMCPClientIntegrationTests against the in-repo scripts/mcp-fixture-server.py; mcp-servers.example.json curates popular OSS servers (filesystem, fetch, github, applescript) as ready-made connectors. Bundled one-tap catalog (Settings → MCP): PaceMCPServerCatalog ships a fixed four-server list (filesystem, fetch, applescript, composio) installable with a click; github/slack/linear now route through the composio OAuth bridge (PaceMCPServerCatalog.supersededBySlug). PaceMCPCatalogInstaller.install/uninstall(_:into:) performs an atomic JSON merge into mcp-servers.json — temp file + rename — that preserves every user-added entry. Catalog is bundled with each Pace release; no remote fetch of the server list at runtime.

Observation, journals, proactivity

Watch mode: PaceScreenWatchModeController is the explicit watch-loop primitive. It samples screenshots, uses PaceScreenImageDiffer, and emits typed events only when a screen has meaningful visual change (majorScreenChange, contentUpdate, focusedRegionChange). The companion panel exposes a Watch Mode toggle, and explicit voice commands such as “watch my screen” / “stop watching” route through PaceWatchModeCommandParser before the planner. Watch events also journal into the local retrieval index (PaceScreenWatchJournal, source screenWatchHistory) so “what did I do today?” questions answer from local history.
Time understanding (journals): two passive retrieval sources power Dayflow-style recall. The screen watch journal records watch-mode events (timestamp, change category, frontmost app, cached VLM description when fresh) into day-per-screen documents; the app usage journal (PaceAppUsageTracker + PaceAppUsageJournal, source appUsageHistory) tracks foreground app minutes and switch counts through permission-free NSWorkspace notifications even when watch mode is off. Both keep 7 days, dedupe/cap entries, rehydrate across restarts, and honor per-source enable/clear in Settings. A third recall source, researchHistory (PaceResearchJournal), captures each research-lane turn as a day-bucketed {question, answer} document (30-day/100-entry retention, same-day dedupe) — recorded by a strictly isResearchTurn-guarded, non-blocking, fire-once hook at the agent-loop turn finalization (question = original transcript, answer = final spoken text) — so “what did I research about X?” answers from history and the turns are browsable in Settings → Memory → Past research.
Proactive features (opt-in): the menu-bar app exposes a small set of proactive surfaces gated by PaceRestraintGate (active-call check, proactive cooldown, intent confidence). Posture watch (PacePostureMonitor), focus-fatigue nudges, calendar pre-meeting nudges, watch-mode observation nudges, and the daily weekday morning brief (PaceMorningTriageScheduler + PaceMorningBriefBuilder) all default OFF and become inert until the user enables them in Settings. When restraint says “stay quiet” the morning brief is parked on a panel card so the user never misses it.
On-device meeting notes: PaceMeetingModeController (in PaceSystemAudioCapture.swift) captures mic + system audio as two separate tracks via PaceMeetingAudioRecorder, segments them into attributed turns via PaceMeetingTurnSegmenter (Accelerate RMS + hysteresis + echo trimming), transcribes each turn on-device via PaceAudioFileTranscriber.transcribeAudioFileSegmented (WhisperKit → Apple Speech fallback), synthesizes structured notes (summary + action items + decisions) via PaceMeetingNotesBuilder using a privacy-pinned LOCAL planner (BuddyPlannerClientFactory.makeLocalOnlyPlannerForPrivacyPinnedFeatures() — never the CLI bridge or Direct API tier, regardless of the user’s tier selection), and journals them via PaceMeetingNotesJournal into the meetingNotes retrieval source so “what did we decide in standup?” answers from local history. Fully on-device — no cloud STT, no cloud LLM. The wedge against Granola/Otter/Fireflies (all cloud). Recording files use .part → atomic rename with RIFF-header crash repair. See PRD docs/product/prds/on-device-meeting-notes.md and ADR decisions/0001-meeting-audio-capture.md. Adaptive note profiles (meetily-informed, Pace-native — see openspec/changes/archive/2026-07-12-adaptive-meeting-notes): notes are shaped by a selectable PaceMeetingNoteProfile (bundled Resources/meeting-note-profiles/<slug>.json — general/standup/one-on-one — plus user overrides in ~/Library/Application Support/Pace/meeting-note-profiles/, loaded/validated by PaceMeetingNoteProfileLibrary). The general profile renders byte-for-byte identical to the legacy prompt (compat anchor, default). Selection precedence: explicit selection (voice command or per-meeting panel pick) → non-general default preference → optional local inference (a silent on-device classify call, default OFF) → general. Voice-triggered: “start my one-on-one recording” / “record this standup” starts a meeting with that profile pinned — PaceMeetingModeCommandParser routes the profile name before the recording starts.
Deeplinks and typed-turn scheduling (pace://): external launchers (Raycast, Shortcuts) can trigger Pace via pace://listen, pace://chat?text=..., pace://watch?enabled=true|false, and pace://panel. PaceDeepLinkParser is a pure reject-on-ambiguity parser (500-char chat cap). Listen remains start-only while a turn is active. Typed chat submitted during an active turn enters PaceChatTurnQueue and drains FIFO as each complete turn returns to idle; the panel keeps optimistic user rows in order, shows the pending count, and can clear pending work without cancelling the active turn. Pace deliberately does not run complete turns in parallel because TTS, approval, conversation memory, and Mac actions share one authoritative agent loop and parallel completion would interleave user-visible effects. Each active turn owns a generation lease plus separately tracked routing and response tasks, so explicit cancellation during intent or automation discovery cannot later install stale speech or UI state. Chat/listen turns still run the normal intent/approval pipeline, so a deeplink can do nothing the user’s own voice couldn’t. URL types registered via CFBundleURLTypes; handled in CompanionAppDelegate.application(_:open:) with pre-launch buffering.

Loop, routing, latency

Intent routing: PaceIntentClassifier is a tiny rule-based local classifier. It routes chitchat to a canned response, pure-knowledge to a text-only planner path, and labels screen-read, tool-action, and phone-large-model turns without paying model latency. On eligible Macs, only unmatched .unknown turns use the Apple Foundation Models classifier as a local fallback before the full pipeline.
Plan-act-observe loop: CompanionManager.sendTranscriptToPlannerWithScreenshot runs a multi-step loop. Each step re-screenshots, re-invokes the VLM (heuristic permitting) and the planner, executes grouped tool calls/actions, returns tool observations to the next planner step, and continues until the planner emits [DONE], emits no tool calls/action tags, or hits AgentMaxSteps (default 8).
Speculative planner race (first step only): on a screen-action / screen-description FIRST step, when CompanionManager.speculativeRaceShouldFire passes (race toggle on, local VLM configured, Apple FM available), performFirstStepSpeculativePlannerRace runs PaceSpeculativePlannerRace.raceSpeculative: the in-process Apple FM “lite” planner (transcript + thread-memory only, NO VLM) races the full VLM-fed planner. The full path’s VLM+OCR+AX prep is deferred into the race’s lazy builder so it runs concurrently with lite — lite produces audio in ~150 ms while a cold VLM (2–3 s) is still resolving. Whichever streams first wins TTS; the full path supersedes the lite stream if its first token lands within 500 ms / 60 spoken chars (prepareForSupersedingStreamWithinTurn resets the pipeline so the new stream replays cleanly). Action parsing ALWAYS uses the full planner’s complete text, never lite — the lite path is spoken-feedback only and can’t emit a real action; when lite wins the audio, the lite text becomes the spoken/bubble/memory string so flushFinal can’t double-speak. bothFailed routes to PaceFailureNarrator. Gate false (and every multi-step turn) keeps the single-planner path byte-identical. Toggle: PaceUserPreferencesStore.enableSpeculativePlannerRace (default ON, opt-out in Settings → Planner).
Walking avatar (optional): PaceAvatarOverlay paints a small SwiftUI character at the bottom of the cursor screen in its own tiny NSPanel, but it is not attached at app launch. The default always-visible surface is the menu-bar/notch capsule.

Key architecture decisions

Native Surface Pattern: PaceOnboardingWindowManager hosts a versioned, resumable first-run state machine that migrates the legacy Boolean completion flag, consolidates live permission setup, and reaches one typed or spoken request through submitChatTranscriptFromDeepLink. A truthful response or completed action contracts the in-window signal toward a notch-shaped anchor before the window closes; failure and blocked outcomes do not play success. PaceMenuBarOverlayManager derives the closed Living Notch frame from NSScreen.safeAreaInsets and the two auxiliary top areas and matches the physical housing exactly at idle. Hover and active states add compact horizontal wings at menu-bar height, with state left of the reserved housing width and signal right of it. A deliberate click resizes that same clipped NSPanel equally left, right, and downward, embedding PacePanelChatView inside the surface instead of opening a visually joined second window. MenuBarPanelManager routes show, hide, shortcut, deep-link, and smoke-test requests into the integrated notch only while reliable physical-notch geometry exists. On notchless or external-only display setups, PaceMenuBarOverlayManager creates a standard NSStatusItem, and the same requests open the standalone panel anchored below it. The geometry and fallback choice recalculate after wake and display changes; Pace never draws a synthetic housing. PaceWindowLayering keeps the optional screen-edge activity glow below both the system menu bar and Living Notch while deliberate teaching annotations remain topmost; the glow defaults off and is transparent while idle. Delayed onboarding transitions are cancellable and generation-guarded so stale work cannot advance a newer scene.

Command Center Pattern: PaceMainWindowManager owns one normal resizable NSWindow for every durable management surface. PaceCommandCenterRouter groups primary destinations under Use Pace, Activity & Privacy, Customize, and Help, while optional integrations and developer controls use one searchable Advanced disclosure. It honors exact requested routes and restores the last valid selection. PaceSettingsWindowManager is a compatibility facade into the same window rather than a second host. The notch panel remains a present-turn surface; its gear routes to Customize → General. Pace still runs as LSUIElement=true, and the Command Center explicitly activates the app only when opened.

Cursor Overlay: A full-screen transparent NSPanel hosts the blue cursor companion. It’s non-activating, joins all Spaces, and never steals focus. Cursor flight, response text placement, and pointing animations render in this overlay via SwiftUI through NSHostingView; listening/thinking animation lives in the notch bar.

Global Push-To-Talk Shortcut: Background push-to-talk uses a listen-only CGEvent tap instead of an AppKit global monitor so modifier-based shortcuts like ctrl + option are detected more reliably while the app is running in the background.

Transient Cursor Mode: When “Show Pace” is off, pressing the hotkey fades in the cursor overlay for the duration of the interaction (recording → response → TTS → optional pointing), then fades it out automatically after 1 second of inactivity.

Local-mode setup

See SETUP_LOCAL.md for the full recipe. The complete Info.plist switch reference (local VLM / planner / TTS-sidecar / transcription-provider knobs and their defaults) lives in ../development/info-plist-switches.md.