--- title: On-device meeting notes description: PRD for local two-track meeting capture, transcription, and structured notes. --- # PRD — On-device meeting notes > **Follow-up (shipped separately):** adaptive, meeting-type-aware note > profiles + transcript-grounded action items — meetily-informed but > Pace-native — are specced in `openspec/changes/adaptive-meeting-notes`. > The `general` profile reproduces this PRD's `{summary, actionItems, > decisions}` output byte-for-byte and remains the default. ## Goal Turn Pace's meeting mode from a stub that publishes RMS levels into a real product: capture mic + system audio as two separate tracks, segment the audio into attributed turns, transcribe on-device, and synthesize structured notes (summary + action items) that feed the existing retrieval index — so "what did we decide in standup?" is answerable from local history. Fully on-device. No cloud STT, no cloud LLM, no bytes off the Mac. This is the headline wedge: Granola, Otter, and Fireflies all upload audio to the cloud. Pace already has the ingredients (SCStream capture, `PaceAudioFileTranscriber` with WhisperKit + Apple Speech backends, `PaceDictationPostProcessor`, the journal/retrieval index). They are unassembled. This PRD is the assembly plan. Blueprint reference: the `os-june` clone's audio pipeline (source-separated capture → energy-based turn segmentation → speaker-echo trimming → structured notes) is the right shape to copy. Pace keeps its in-process SCStream rather than os-june's out-of-process CoreAudio tap (see ADR candidate below). ## Scope (v1) A meeting is started and stopped explicitly (voice command or panel toggle). While active: 1. **Two-track capture** — mic via `AVAudioEngine`, system via the existing `SCStream` in `PaceSystemAudioCapture`. Two independent `PaceMeetingAudioTrack` buffers, never mixed. Enables "you" vs "them" attribution and dropping a silent system track (no meeting audio → it's a solo dictation, not a meeting). 2. **On-device recording** — each track is written to a `.wav` (RIFF) under `~/Library/Application Support/Pace/meetings//`. Partial file with a `.part` suffix during recording; atomic rename to final on clean stop. RIFF-header repair on crash-recovery scan at launch so a force-quit mid-meeting still yields a playable file. 3. **Energy-based turn segmentation** — Accelerate RMS over fixed-size windows + a hysteresis state machine (speech / silence / speech) produces timestamped turn boundaries per track. Speaker-echo trimming: when the mic and system tracks both have energy inside a window, the louder track owns the turn and the other is suppressed for that window (kills Pace's TTS echo and Zoom echo). Output is a `[PaceMeetingTurn]` array with `start`, `end`, `track`, `attributedSpeaker`. 4. **Transcription** — each turn's audio slice is fed to `PaceAudioFileTranscriber` (WhisperKit first, Apple Speech fallback). The transcriber is extended to return per-segment text + timestamps instead of one concatenated string, so turns become attributable transcript lines. `PaceDictationPostProcessor` runs over the joined transcript for capitalization/punctuation cleanup. 5. **Notes synthesis** — the transcript is sent to the active planner (LM Studio `qwen3-30b-a3b` by default; Apple FM for short meetings if available) with a focused `PaceMeetingNotesPrompt` that returns JSON: `{summary, actionItems: [{text, owner, due}], decisions: [...], attendees: [...]}`. Failures fail loud (no silent fallback) per the existing planner posture; the raw transcript is still saved as the artifact so the user loses nothing. 6. **Retrieval wiring** — the notes + transcript are journaled as a new `PaceRetrievalSource.meetingNotes` document so "what did we decide in standup?" / "did we agree on the launch date?" answer from local history. Day-bucketed like `PaceScreenWatchJournal`. Out of scope for v1: real-time streaming transcription (turns are transcribed after stop, not live), speaker diarization beyond mic-vs-system attribution, multi-participant naming, calendar event auto-association (the user names the meeting on stop), cross-meeting action-item tracking, cloud sync. Just: capture → segment → transcribe → notes → recall. ## Architecture ### Modify: `leanring-buddy/PaceSystemAudioCapture.swift` `PaceMeetingModeController` today throws audio away — `handleAudioSample` only forwards an RMS level. Extend the delegate to also append the raw Float32 samples to a `PaceMeetingAudioTrack` (system) buffer, and expose a `systemAudioTrack: PaceMeetingAudioTrack?` property. The existing RMS publisher stays (the panel sound animation still uses it). Add a `recordingURL: URL?` property and write the track to disk on `stop()`. ### New file: `leanring-buddy/PaceMeetingAudioRecorder.swift` (~250 lines) Owns the mic track via `AVAudioEngine` + input node tap, and the two-track disk writer. Public API: ```swift @MainActor final class PaceMeetingAudioRecorder { let meetingID: UUID private(set) var micTrack: PaceMeetingAudioTrack? private(set) var systemTrack: PaceMeetingAudioTrack? private(set) var recordingDirectoryURL: URL init(meetingID: UUID, now: Date = Date()) func startMicCapture() async throws func appendSystemSamples(_ samples: [Float]) // called by the SCStream delegate func stop() async -> PaceMeetingRecording // flushes both tracks, atomic renames func crashRepairIfNeeded() // called at launch; repairs .part files } ``` `PaceMeetingAudioTrack` is a value type: `[Float]` samples + sample rate + channel count. The writer serializes RIFF WAV headers (16-bit PCM) directly — no AVAudioFile dependency for the recording path so a mid-write crash leaves a header that `crashRepairIfNeeded` can patch (recompute chunk sizes from the file length). ### New file: `leanring-buddy/PaceMeetingTurnSegmenter.swift` (~200 lines) Pure function over two tracks. No I/O, no async — unit-testable in isolation. Uses Accelerate `vDSP_rmsqv` over 20 ms windows with a hysteresis state machine (silence threshold 0.01, speech threshold 0.04, min turn 600 ms, max silence gap 400 ms to merge adjacent speech). Echo trimming: when both tracks exceed the speech threshold in the same window, the louder track wins and the other is marked silent for that window. ```swift nonisolated enum PaceMeetingTurnSegmenter { static func segment(mic: PaceMeetingAudioTrack?, system: PaceMeetingAudioTrack?, now: Date) -> [PaceMeetingTurn] } struct PaceMeetingTurn: Equatable { let start: Date let end: Date let track: PaceMeetingAudioTrackKind // .mic (you) / .system (them) let attributedSpeaker: String // "you" / "them" for v1 let sampleRange: Range // into the owning track's buffer } ``` ### Modify: `leanring-buddy/PaceAudioFileTranscriber.swift` Add a second entry point that returns timestamped segments instead of one concatenated string. The existing `transcribeAudioFile(at:)` stays for the App Intent path (drag-drop result text wants plain text). ```swift struct PaceTranscriptionSegment: Equatable { let start: TimeInterval let end: TimeInterval let text: String } static func transcribeAudioFileSegmented(at fileURL: URL) async throws -> [PaceTranscriptionSegment] ``` WhisperKit already returns segments — `transcribeWithWhisperKit` joins them today; the new path returns them directly. Apple Speech fallback returns a single segment spanning the whole file (it has no segment API), so segmented callers degrade to one block on the fallback path. Document this in the doc comment. ### New file: `leanring-buddy/PaceMeetingNotesBuilder.swift` (~220 lines) Drives the planner to produce structured notes from the transcript. Pure-ish: takes a transcript + the active `BuddyPlannerClient`, returns a `PaceMeetingNotes` value. Parses the planner's JSON response with a lenient decoder (unknown fields ignored, missing action items → empty array, not a crash). On planner failure, returns the raw transcript as the "summary" with `synthesisFailed: true` so the artifact is still saved and recallable. ```swift struct PaceMeetingNotes: Equatable, Codable { let meetingID: UUID let startedAt: Date let endedAt: Date let title: String // user-supplied or "Meeting