Audio pipeline

How Pace hears and speaks — capture, streaming synthesis, interruption, and the two-track meeting recorder. The ASR/TTS models live in new-things.md (WhisperKit, Kokoro/mlx-audio); this page is the plumbing around them.

Push-to-talk via CGEvent tap

What: A listen-only CGEvent tap that detects the ctrl+option push-to-talk chord system-wide, even when Pace has no focus.
Why here: An AppKit global monitor misses modifier-only combos while backgrounded; the low-level event tap catches the press reliably so hold-to-talk always works.
Where: GlobalPushToTalkShortcutMonitor.swift — GlobalPushToTalkShortcutMonitor (the notch chat shortcut mirrors it in GlobalChatShortcutMonitor.swift).
Source: https://developer.apple.com/documentation/coregraphics/cgevent/1454426-tapcreate

Streaming sentence TTS pipeline

What: Consumes the planner’s token stream, cuts it at sentence boundaries, and queues each finished sentence to TTS while later sentences are still generating.
Why here: The core sub-500 ms TTFSW trick — speech starts on sentence 1 while the planner is still writing sentence 2; sentence N+1 is synthesized while N plays.
Where: StreamingSentenceTTSPipeline.swift — StreamingSentenceTTSPipeline.
Source: internal — no external spec.

Barge-in VAD

What: Detects the user talking over the assistant using an RMS energy threshold plus echo suppression (a grace window and a raised threshold while TTS is playing).
Why here: Lets you interrupt Pace mid-reply without the assistant’s own TTS bleed tripping a false interruption.
Where: PaceBargeInVAD.swift — PaceBargeInVAD / PaceBargeInVADConfiguration.
Source: internal — no external spec.

Wake word spotter

What: Always-listening keyword spot via Apple SFSpeechRecognizer with requiresOnDeviceRecognition=true (ANE-backed), pausing during push-to-talk and honoring low-power / screen-sleep.
Why here: Opt-in hands-free trigger that persists nothing to disk — the on-device recognizer is the same one used for dictation, so no extra model ships.
Where: PaceAppleSpeechWakeWordSpotter.swift — PaceAppleSpeechWakeWordSpotter (behind PaceWakeWordSpotterProtocol).
Source: https://developer.apple.com/documentation/speech/sfspeechrecognizer

Two-track meeting capture

What: Records mic (AVAudioEngine tap) and system audio (ScreenCaptureKit SCStream, excluding Pace’s own process) as two separate mono WAV tracks with per-track start offsets.
Why here: Separate tracks give clean speaker attribution (“you” vs “them”) for on-device meeting notes — better than one mixed track, and it excludes Pace’s own TTS to avoid self-capture.
Where: PaceSystemAudioCapture.swift — PaceMeetingModeController; PaceMeetingAudioRecorder.swift — PaceMeetingAudioRecorder. ScreenCaptureKit itself → see new-things.md.
Source: internal (built on ScreenCaptureKit + AVAudioEngine).

Off-main-actor track writer (FIFO)

What: A per-track AsyncStream fed by the capture callbacks and drained by a single detached consumer that does the Float32→PCM16 conversion and FileHandle writes off the main actor, in exactly the produced order.
Why here: Replaces per-buffer Task { @MainActor } hops that had no ordering guarantee (buffers could interleave in the WAV) and ran disk I/O on the main thread; the single-consumer stream makes ordering correct by construction.
Where: PaceMeetingAudioRecorder.swift — MeetingTrackWriter / MeetingTrackFinalState; the mic conversion is boxed in MicSampleConverter.
Source: internal — no external spec. (Swift concurrency idioms → infra-and-patterns.md.)

Meeting turn segmentation

What: Splits the two tracks into attributed turns using per-window RMS (Accelerate), hysteresis on speech/silence thresholds, and echo trimming when both tracks fire at once.
Why here: Turns are the unit the meeting-notes builder summarizes; hysteresis prevents a single pause from shattering one utterance into many fragments.
Where: PaceMeetingTurnSegmenter.swift — PaceMeetingTurnSegmenter, PaceMeetingTurn, PaceMeetingAudioTrack.
Source: https://developer.apple.com/documentation/accelerate (RMS math); hysteresis + echo-trim are internal.

Segmented audio-file transcription

What: Transcribes each meeting turn on-device via WhisperKit for long audio, falling back to Apple Speech (which caps ~1 minute) per segment.
Why here: Meetings routinely exceed Apple Speech’s limit, so WhisperKit handles the bulk while Apple Speech guarantees a result on any Mac.
Where: PaceAudioFileTranscriber.swift — PaceAudioFileTranscriber.transcribeAudioFileSegmented. WhisperKit itself → see new-things.md.
Source: internal (orchestration over WhisperKit + Apple Speech).

RIFF-header crash repair

What: Recording files are written as .part with a placeholder WAV header, patched with real chunk sizes and atomically renamed on clean stop; a launch-time sweep repairs any .part left by a crash.
Why here: A force-quit mid-meeting would otherwise leave an unplayable file with a zero-length header — the repair sweep recovers the audio instead of losing it.
Where: PaceMeetingAudioRecorder.swift — crashRepairAllMeetingRecordings(), repairRIFFPartFile(...), riffHeaderPlaceholder(...).
Source: https://en.wikipedia.org/wiki/WAV (RIFF/WAVE container).