Offline voice-to-structured meeting notes

The idea

A React Native app that records a meeting or voice memo, runs it through Whisper.cpp on-device, and formats the transcript into structured notes — key decisions, action items, and open questions. No audio leaves the phone. The user gets a clean Markdown file they can paste anywhere.

Why build this

Cloud meeting recorders like Otter.ai and Fireflies are non-starters in legal, finance, healthcare, and any organization with data residency requirements. The missing piece has always been capable on-device transcription; that gap closed when Whisper shrank enough to run on mobile CPUs and Apple's Core ML framework made it practical on recent iPhones.

There is real demand from consultants, lawyers, and executives who sit in confidential rooms and still want notes. They currently have two options: take notes by hand or send sensitive audio to a third party. A polished offline-first app is a third option that doesn't exist yet at the quality level users expect.

Stack sketch

• React Native (Expo) for iOS and Android from a single codebase
• whisper.rn — React Native bindings for Whisper.cpp — for on-device transcription; tiny.en for speed, base.en as the default, medium.en as an optional quality upgrade
• A short structuring prompt sent to Claude API (user supplies their own key via settings) to turn the raw transcript into sections; falls back to a regex-based heuristic if no key is set
• expo-av for audio capture and expo-file-system for model weight management
• SQLite via expo-sqlite for local note storage
• expo-sharing for Markdown and plain-text export

Scope for v1

• Record audio and transcribe fully on-device
• Produce a structured note: one-paragraph summary, bulleted action items, open questions
• Store notes locally with title, date, and recording duration
• Export as Markdown or copy to clipboard
• iOS first; ship Android once the Core ML bindings stabilize

Out of scope for v1: speaker diarization, calendar integration, team sync, custom templates, any cloud storage.

Where it could go

Speaker diarization — labeling who said what — is the most-requested feature for meeting note tools and the hardest to do on-device. Pyannote.audio has a distilled mobile-friendly model; integrating it turns the transcript from a wall of text into a readable dialogue and justifies a paid tier on its own.

Once per-speaker attribution works, the next step is a team sync layer: encrypted export to a self-hosted server (one Docker container) so assistants can pull notes from an exec's phone, search across meetings by project or person, and surface recurring action items that were never closed. That architecture satisfies enterprise no-cloud requirements while adding the collaboration surface a paying team actually needs.

Watch out for

Whisper model weights are large — base.en is 145 MB, medium.en is 1.5 GB — so ship the app without bundled weights and download the selected model on first launch over Wi-Fi, or Apple will reject the binary for exceeding the cellular download limit.