macOS Menu Bar App & Chrome Companion Extension Guide

OpenScribe splits local capture on macOS across two components that communicate strictly over loopback (127.0.0.1:48721):

1. OpenScribe Chrome Companion Extension (extension/): Runs inside Google Chrome on https://meet.google.com/* (extension/manifest.json). It hooks WebRTC RTCRtpReceiver instances (extension/src/main-runtime.js and extension/src/opus-pipeline.js), decodes per-stream Opus packets to 48,000 Hz mono PCM using WebCodecs AudioDecoder, tracks RTP contributing sources (CSRCs) and participant device IDs, and forwards metadata plus 48 kHz PCM chunks to 127.0.0.1:48721 through extension/background.js.
2. OpenScribe.app (native/): A native macOS 14+ menu bar and companion window application written in Swift 6 (AppKit + SwiftUI). It listens on 127.0.0.1:48721 (LocalHostBridgeServer.swift), records your local voice (local_mic) at 48,000 Hz via the outgoing Meet audio track or LocalMicrophoneCapturer.swift, writes crash-safe local recordings to ~/Library/Application Support/OpenScribe/recordings/, streams live audio through TranscriptionStreamManager.swift (elevenlabs, openai_realtime, gemini_realtime, and openai_compatible), and syncs transcripts, active-meeting notes, links, files, and screenshots to your account.

1. Platform Capture Capabilities & Honest Verification Status

OpenScribe records the verified captureCapability and speaker attribution mode on every meeting and transcript segment (native/Sources/OpenScribeCore/Models.swift and server/API.md). It never fabricates speaker names or emails when a platform only provides mixed audio or window metadata:

2. Building and Loading the Chrome Companion Extension (extension/)

Step 1: Bundle the Extension Scripts

From the repository root, run the extension build script (extension/build.mjs):

node extension/build.mjs

extension/build.mjs concatenates the plain JavaScript source modules in extension/src/ (meet-collections.js, opus-pipeline.js, schema.js, main-runtime.js, and isolated-runtime.js) into extension/meet-main.js (MAIN world content script) and extension/meet-isolated.js (ISOLATED world content script) directly inside extension/ alongside extension/manifest.json, extension/background.js, extension/popup.html, and extension/popup.js.

Step 2: Load Unpacked in Google Chrome

1. Open Google Chrome and navigate to chrome://extensions.
2. Enable Developer mode using the toggle in the top-right corner.
3. Click Load unpacked in the top-left corner.
4. Select the /path/to/openscribe/extension directory (do not look for a dist/ subdirectory, because manifest.json and the bundled .js files live directly in extension/).
5. Confirm that OpenScribe Meet Companion (version 0.1.0) appears and is enabled.

How the Extension Communicates with OpenScribe.app (LocalHostBridgeServer.swift)

The extension never talks to your remote OpenScribe server directly and holds no cloud credentials. Instead, extension/background.js connects only to http://127.0.0.1:48721 (native/Sources/OpenScribeCore/LocalHostBridgeServer.swift):

1. Bridge Readiness Check (GET /api/status):
   • You can check whether OpenScribe.app is listening on port 48721 at any time by running:
     bash

     curl -sS http://127.0.0.1:48721/api/status

   • Expected 200 OK JSON response (RecorderCoordinator.swift):
     json

     {
       "ok": true,
       "app": "OpenScribe",
       "port": 48721
     }

2. Loopback Pairing (GET /api/extension/pair or POST /api/extension/pair):
   • extension/background.js requests a random bridge session token from http://127.0.0.1:48721/api/extension/pair.
   • LocalHostBridgeServer.handlePair requires a non-empty Origin header matching the pinned extension origin chrome-extension://bnfgpacpapkllohejfcahnblgkodahnb (expectedExtensionId). Requests with an empty Origin return 403 {"error": "extension_origin_required"}, and requests from any other origin return 403 {"error": "unauthorized_origin"}.
   • When the origin matches, LocalHostBridgeServer returns 200 OK:
     json

     {
       "ok": true,
       "token": "<bridgeAuthToken>",
       "pairedOrigin": "chrome-extension://bnfgpacpapkllohejfcahnblgkodahnb",
       "port": 48721
     }

3. WebSocket & HTTP Event Streaming (/ws/extension and POST /api/extension/event):
   • After pairing, extension/background.js opens a WebSocket to ws://127.0.0.1:48721/ws/extension?token=<bridgeAuthToken> and sends an authentication frame on open:
     json

     {
       "type": "authenticate",
       "bridgeToken": "<bridgeAuthToken>"
     }

   • LocalHostBridgeServer replies with {"type": "authenticated", "ok": true} and accepts incoming ExtensionIncomingMessage frames (Models.swift and RecorderCoordinator.swift) with type values "presence", "roster", "audio_batch", "capture_error", and "status".
   • If the WebSocket is not currently open, extension/background.js falls back to POST http://127.0.0.1:48721/api/extension/event with header X-OpenScribe-Bridge-Token: <bridgeAuthToken> (or Authorization: Bearer <bridgeAuthToken>), which returns {"ok": true}.

3. Building and Launching OpenScribe.app (native/)

Prerequisites

• macOS 14.2 (Sonoma) or newer
• Xcode Command Line Tools (xcode-select --install) with Swift 6 (swift --version)

Build the Application Bundle (native/scripts/build-app.sh)

Run the packaging script from the repository root:

./native/scripts/build-app.sh

What native/scripts/build-app.sh does:

1. Compiles the release binary (swift build -c release --product OpenScribe) inside native/.
2. Creates the macOS application bundle at ./native/dist/OpenScribe.app (or at <custom-dir>/OpenScribe.app if you pass a target directory argument such as ./native/scripts/build-app.sh /Applications).
3. Writes Contents/Info.plist (CFBundleIdentifier = com.akyonlabs.openscribe, NSMicrophoneUsageDescription, NSScreenCaptureUsageDescription) and ad-hoc signs the bundle (codesign --force --deep --sign -).

Launch the built app in Finder or from the terminal:

open ./native/dist/OpenScribe.app

Or run the binary directly in the terminal to view structured stdout/stderr logs:

./native/dist/OpenScribe.app/Contents/MacOS/OpenScribe

Supported Command-Line Arguments (OpenScribeMain.swift)

OpenScribe supports three non-interactive CLI modes for automated verification and testing (native/Sources/OpenScribeApp/OpenScribeMain.swift):

OpenScribe does not take --backend-url or --bridge-port CLI flags. Change the workspace in Settings. Development runs can set OPENSCRIBE_SERVER_URL before launch.

4. Pointing OpenScribe.app to Your Server (BackendClient.swift)

A fresh app connects to https://scribe.akyonlabs.com. BackendClient (native/Sources/OpenScribeCore/BackendClient.swift) preserves an explicitly configured workspace. It resolves the server base URL in the following precedence order:

1. OPENSCRIBE_SERVER_URL environment variable (if set and non-empty when the process starts).
2. UserDefaults key serverBaseURL in domain com.akyonlabs.openscribe.
3. Default fallback: https://scribe.akyonlabs.com.

Change the workspace

1. Open Settings with the gear button or Cmd+,.
2. Under Workspace, click Change Workspace….
3. Enter the workspace address and click Connect.

The app validates the workspace before switching. Credentials and local recordings stay bound to their originating workspace. Sign in to the selected workspace through browser pairing.

For development, OPENSCRIBE_SERVER_URL sets the starting address. Automated tests can supply OPENSCRIBE_SESSION_TOKEN through the process environment. Never put a session token in a URL or command log.

Choose calendars and open meetings

Connect calendars in the web workspace. In Mac Settings, check the calendars you want to see under each connected account. Selections are saved for that workspace and signed-in user.

Upcoming groups events by local date and shows the time, owning account and meeting-platform icon. Join opens the meeting link. Open a row for its calendar and attendees. The calendar account identifies the event owner, not a verified account used to join the call.

Past opens each meeting in the web workspace. During an active call, Live contains the transcript and saved notes. Browsing other tabs preserves recording and the composer draft.

5. Browser Device Pairing & Opt-In macOS Keychain Storage

OpenScribe.app never embeds Google OAuth client secrets. Instead, it pairs with your OpenScribe server through your web browser (native/Sources/OpenScribeCore/BackendClient.swift, RecorderCoordinator.swift, and server/auth.go).

Device Pairing Flow (/api/openscribe/auth/device/*)

1. Start Pairing (POST /api/openscribe/auth/device/start):
   • Click Sign In in the OpenScribe companion window (CompanionView.swift) or inside Settings -> Account (SettingsView.swift).
   • BackendClient.startDeviceLogin sends POST <serverBaseURL>/api/openscribe/auth/device/start with {"device_label": "<Mac Host Name>"}.
   • The server creates a pending record in the device_logins collection (valid for 10 minutes) and returns:
     json

     {
       "id": "<request_id>",
       "code": "ABCD-EFGH",
       "user_code": "ABCD-EFGH",
       "secret": "<high_entropy_secret>",
       "device_code": "<high_entropy_secret>",
       "verify_url": "https://scribe.example.com/pair?id=<request_id>&code=ABCD-EFGH",
       "verification_uri": "https://scribe.example.com/pair",
       "verification_uri_complete": "https://scribe.example.com/pair?id=<request_id>&code=ABCD-EFGH",
       "expires": "2026-11-09T14:30:00Z",
       "expires_in": 600,
       "interval": 2
     }

2. Approve in Browser (/pair -> POST /api/openscribe/auth/device/approve):
   • OpenScribe.app displays the short verification code (such as ABCD-EFGH) and opens verify_url in your default browser.
   • Sign in with Google if prompted, confirm the verification code matches what is shown in OpenScribe.app, and click Approve Device.
   • The web app sends POST /api/openscribe/auth/device/approve with {"id": "<request_id>", "code": "ABCD-EFGH"} and Authorization: Bearer <web_jwt>, returning {"status": "approved"}.
3. Poll for Token (POST /api/openscribe/auth/device/poll):
   • RecorderCoordinator.pollDeviceApprovalLoop polls POST <serverBaseURL>/api/openscribe/auth/device/poll with {"id": "<request_id>", "secret": "<high_entropy_secret>"} every 2 seconds (up to 90 attempts, or 3 minutes).
   • Once approved, the server returns {"status": "approved", "token": "<pb_auth_jwt>", "user": {"id": "...", "email": "...", "name": "...", "admin": ...}} and immediately wipes issued_token from device_logins so the poll secret cannot be reused.

Opt-In macOS Keychain Storage (KeychainStore.swift)

To avoid unexpected macOS Keychain password prompts during development or automated runs, persistent Keychain storage is opt-in (rememberSignIn defaults to false in RecorderCoordinator.swift):

• Default (rememberSignIn = false): The session JWT is held only in memory for the current OpenScribe.app process. No Keychain read or write calls are made on startup, and KeychainStore.clearSessionToken() removes any previously saved item when toggled off or when signing out.
• Opt-In (rememberSignIn = true): Open Settings -> Account and enable Remember sign-in on this Mac (SettingsView.swift). When enabled, KeychainStore.saveSessionToken(_:) stores the token in the macOS login Keychain with these attributes:
  • kSecClass: kSecClassGenericPassword
  • kSecAttrService: "com.akyonlabs.openscribe"
  • kSecAttrAccount: "app_session_token"
  • kSecAttrLabel: "OpenScribe sign-in"

KeychainStore sets SecKeychainSetUserInteractionAllowed(true) only during interactive saveSessionToken(_:) calls and sets it to false during loadSessionToken() and clearSessionToken() so background checks never block on a modal prompt.

To delete a stored Keychain token from the terminal:

security delete-generic-password -s "com.akyonlabs.openscribe" -a "app_session_token"

6. macOS permissions

OpenScribe requests access when you use the feature that needs it. Settings shows actions for missing access.

Install the app in Applications before granting access. Development builds use ad-hoc signing, so rebuilding changes their code identity. An enabled entry for an older build can remain visible while macOS rejects the current build. Accessibility and Screen Recording are separate grants. The app checks actual access rather than treating the visible toggle as proof.

7. Active-Meeting Notes, Links & Files vs. Post-Meeting Web Editing

OpenScribe.app enforces a strict boundary between in-meeting capture and post-meeting curation (RecorderCoordinator.swift and CompanionView.swift):

• During a confirmed joined or recording meeting: Add notes, links, files or meeting-window screenshots. Each accepted item is saved locally with immutable session, backend meeting and capture-time binding before its saved indicator appears. Queued uploads continue to that meeting after the call ends. Notes open as selectable text with Copy and Done, and attachments open in Quick Look.
• After the meeting ends: New Mac additions stop. Open dashboard returns to the web workspace, and each Past meeting opens its exact web detail page at /?meeting=<id>. Add or edit post-meeting content there.

The actual native picker saved the supplied test file locally. Backend attachment receipt and drag-and-drop acceptance remain pending. Isolated SwiftUI actions verified note previews, file Quick Look and joined-call preservation after Stop.

8. Crash-Safe Local Recording & Recovery (LocalRecordingStore.swift)

OpenScribe writes all audio, transcript segments, notes, and attachments to local disk before attempting any cloud upload (native/Sources/OpenScribeCore/LocalRecordingStore.swift). If your network drops, the server restarts, or OpenScribe.app terminates mid-meeting, your recording remains intact on your Mac.

Local Storage Layout

By default, sessions are stored under:

~/Library/Application Support/OpenScribe/recordings/<sessionId>/
├── manifest.json                 # LocalSessionManifest (state, participants, streams, unsyncedSegmentsCount)
├── transcript.jsonl              # Append-only JSON Lines log of partial and committed TranscriptSegments
├── notes.jsonl                   # Append-only JSON Lines log of active-meeting notes and pasted links
├── attachments.jsonl             # Append-only JSON Lines log of active-meeting dropped file metadata
├── attachments/                  # Copied user-dropped files captured during an active meeting
├── screenshots/                  # Captured meeting window PNG files (shot-<timestamp>.png)
└── streams/                      # Per-stream 48,000 Hz mono audio tracks
    ├── <streamKey>.pcm           # Incremental 16-bit little-endian PCM (flushed + fsynced per chunk)
    ├── <streamKey>.f32           # Incremental 32-bit IEEE float PCM
    ├── <streamKey>.packets.jsonl # Per-chunk RTP/observer timestamps and sequence metadata
    └── <streamKey>.wav           # Standard 48 kHz mono WAV file built on finalize or crash recovery

Recovery and sync

At startup, OpenScribe rebuilds missing WAV files from retained PCM and reloads local sessions. It retries pending transcript, note and attachment uploads automatically. Every retry uses the originating meeting and workspace identity.

Past meetings appear in the main window. Clicking one opens its web meeting page after required sync and sign-in. Settings contains preferences and connected calendars.