feat(webrtc): end-to-end encrypted voice & video calls with adaptive codecs

Add 1:1 voice and video calling over the existing SAS-verified peer connection. Audio and video tracks ride the same RTCPeerConnection as the chat, bundled onto one DTLS-SRTP transport, so media inherits the session's end-to-end encryption. SDP offer/answer is renegotiated in-band over the verified data channel — no signalling server, so the media's DTLS fingerprints are authenticated end-to-end. Calls are gated on a connected, SAS-verified session. Codecs & adaptation: - Opus tuned for lossy links (in-band FEC, DTX, RED redundancy); audio is bandwidth-prioritised and never throttled. - VP9/AV1 single-encoding SVC with H.264/VP8 fallback; video degrades by spatial/temporal layer. - Runtime NetworkAdaptationController trims video bitrate on loss/RTT and recovers as the link clears — no renegotiation. Live connection-quality indicator (Excellent/Good/Fair/Weak) in the call UI. In-call controls: mute, camera on/off (voice→video upgrade in-band), camera flip, minimize-to-widget, hang up, and accept/decline for incoming calls. Production logging disabled (DEBUG_MODE=false); temporary call diagnostic logger removed. Codec rationale in docs/webrtc-config.md.
2026-07-23 12:56:19 -04:00
parent 0de8ab2d54
commit b3fcf54670
32 changed files with 3855 additions and 86 deletions
@@ -0,0 +1,143 @@
+# WebRTC Audit — SecureBit.chat call stack
+
+**Step 1 deliverable. No code changed.** This maps the existing WebRTC/call code so we can plan the adaptive voice/video stack (Opus FEC/DTX/RED, VP9-SVC + fallbacks, TWCC/NACK/PLI, getStats adaptation) *without* creating a parallel branch.
+
+---
+
+## 0. Reality vs. the task spec (read this first)
+
+The task is written against a modular TypeScript layout (`src/webrtc/codecs/*.ts`, `call.ts`, `config.ts`, …). **That layout does not exist and does not match the repo.** Concretely:
+
+| Task assumption | Actual repo |
+|---|---|
+| TypeScript (`.ts`) | Plain JS / JSX. No `tsconfig`, no `.ts` files, esbuild bundles JS as-is. |
+| `src/webrtc/` modular stack | One monolith: `src/network/EnhancedSecureWebRTCManager.js` (~14.8k lines). |
+| Fresh `RTCPeerConnection` per call, `addTransceiver` at init | **One long-lived PC** shared with the encrypted **data channel**; calls are **renegotiated onto it**. Media is added with `addTrack`, **never** `addTransceiver`. |
+| Standard signalling server / offer at init | **Two separate SDP paths** (see §5). Call SDP is exchanged **in-band over the E2E data channel**. |
+| Playwright available | Not installed. Tests are plain `node tests/*.test.mjs`. No `RTCPeerConnection` in Node. |
+| `debug('webrtc:adapt')` | No `debug` dependency. Logging is `_secureLog(...)` + the new `window.sbCallLog(...)`. |
+
+**Consequence:** we integrate into the existing manager + a small set of **new plain-JS helper modules** it imports. We do **not** add TypeScript or a `src/webrtc/` TS tree. Proposed JS layout in §7.
+
+---
+
+## 1. Module map
+
+| File | Kind | Exports / globals | Role |
+|---|---|---|---|
+| `src/network/EnhancedSecureWebRTCManager.js` | ES module | `export { EnhancedSecureWebRTCManager, SecureMasterKeyManager, SecureIndexedDBWrapper, SecurePersistentKeyStorage }` (line 14812). Also `window.EnhancedSecureWebRTCManager`. | **All** transport + crypto + call logic. The class to extend. |
+| `src/components/ui/CallUI.jsx` | ES module (side-effect) | `window.CallUIComponent`, `export { CallUIComponent }` | Presentational call overlay (voice/video/minimized/incoming). Pure UI; no media/crypto. |
+| `src/scripts/app-boot.js` | bundled entry | imports the manager + UI components; sets `window.*` | Real loader (→ `dist/app-boot.js`). CallUI is imported here. |
+| `src/app.jsx` | bundled entry | React app (`h`/createElement) | Header call buttons + mounts `CallUI`. getUserMedia for **voice messages** (unrelated to calls) at 816–817. |
+| `config/ice-servers*.js` | script | `window.SECUREBIT_ICE_SERVERS` | Operator STUN/TURN override. |
+| `src/network/iceServers.js`, `iceSettingsStore.js` | ES modules | ICE validation/persistence | User-supplied ICE servers. |
+
+> `src/scripts/bootstrap-modules.js` exists but is **dead** (not referenced anywhere). Ignore it.
+
+---
+
+## 2. RTCPeerConnection: creation & config
+
+- **`createPeerConnection()`** — `EnhancedSecureWebRTCManager.js:7731`. Single `new RTCPeerConnection(config)` at **7737**. Wires `onconnectionstatechange`, `oniceconnectionstatechange`, `onicecandidateerror`, **`ontrack`** (7802, call media), `ondatachannel`.
+  - Called from **9983** (initiator/offer path) and **10646** (responder/answer path).
+- **`_buildPeerConnectionConfig()`** — `7661`. Returns:
+  ```js
+  { iceServers, iceCandidatePoolSize: 10, bundlePolicy: 'balanced' }   // + iceTransportPolicy:'relay' in privacy mode
+  ```
+  No `rtcpMuxPolicy`, no codec/degradation config (nothing to do there; those live on senders/transceivers).
+- One PC per session; multi-session keeps a `Map<id, manager>` (each with its own PC).
+
+## 3. SDP formation
+
+No codec munging exists today — only **DTLS fingerprint extraction** (`_extractDTLSFingerprintFromSDP`) and **ICE candidate diagnostics** (`_summarizeIceCandidatesInSDP` 1057, `_logIceCandidateDiagnostics` 1119, `_warnIfRemoteCandidatesNeedRelay` 1145).
+
+| Op | Line(s) | Path |
+|---|---|---|
+| `createOffer` | 10004 | **Data-channel handshake** (initial connection) |
+| `setLocalDescription(offer)` | 10009 | handshake |
+| `setRemoteDescription(offer)` | 10682 | handshake |
+| `createAnswer` | 10717 | handshake |
+| `setLocalDescription(answer)` | 10727 | handshake |
+| `setRemoteDescription(answer)` | 11462 | handshake |
+| `createOffer` | 13047, 13262 | **Call** (startCall / renegotiate) |
+| `setLocalDescription` | 13048, 13263 | call |
+| `setRemoteDescription(offer)` | 13081 | call |
+| `createAnswer` + `setLocalDescription` | 13087–13088 | call |
+| `setRemoteDescription(answer)` | 13285 | call |
+| `setLocalDescription({type:'rollback'})` | 13176 | call teardown |
+
+**Where codec/RTP munging must hook:** only the **call** offers/answers (13047/13087/13262). The handshake SDP (m=application/data channel only) must stay untouched.
+
+## 4. Media: getUserMedia / addTrack / transceivers
+
+- **getUserMedia (calls):** `_acquireLocalMedia` 13012, `upgradeToVideo` 13221, `switchCamera` 13245.
+  - Unrelated: voice-message recorder `app.jsx:817`; QR scanner `QRScanner.js:72`.
+- **addTrack:** `_acquireLocalMedia` 13017, `upgradeToVideo` 13230. **← the only way media is attached today.**
+- **replaceTrack:** upgrade 13228, switchCamera 13252, teardown 13151.
+- **removeTrack / getReceivers / getTransceivers / .stop():** teardown 13146–13158.
+- **`addTransceiver`: NONE.** `sendEncodings`: NONE. `setParameters`/`getParameters`: NONE. `setCodecPreferences`/`getCapabilities`: NONE.
+- **`ontrack`** handler: 7802 (accumulates into `this.remoteMediaStream`).
+
+## 5. Signalling model (two SDP paths — important)
+
+1. **Connection handshake** (data channel): offer/answer compressed and exchanged **out-of-band** (QR / copy-paste), authenticated by **SAS**. Establishes the DTLS transport + `securechat` data channel (`createDataChannel` 9986).
+2. **Call setup** (media): after the channel is verified, call SDP rides **in-band over that data channel** via `MESSAGE_TYPES.CALL_OFFER/CALL_ANSWER/CALL_ICE/CALL_DECLINE/CALL_END`. Routed in the live `dataChannel.onmessage` (~7996) → `_handleCallSignal` (13274). Media BUNDLEs onto the existing transport, so **no new ICE**.
+
+→ Adaptive stack changes affect **path 2 only**.
+
+## 6. Call subsystem inventory (all in the manager, added recently)
+
+State/getters: `callState` object; `getCallState` 12957, `getRemoteMediaStream` 12961, `getLocalMediaStream` 12965, `_updateCallState` 12969 (fires `onCallStateChanged` + `securebit-call-state` DOM event).
+Lifecycle: `_callCanStart` 12983 (gated on connected+verified), `_acquireLocalMedia` 12999, `startCall` 13025, `_onIncomingCallOffer` 13065, `_answerCallOffer` 13080, `acceptCall`/`declineCall`, `endCall` 13120, `_teardownCallMedia` 13132.
+Controls: `setMicEnabled` 13186, `toggleMic` 13193, `setCameraEnabled` 13197, `toggleCamera` 13215, `upgradeToVideo` 13218, `switchCamera` 13241, `_renegotiateCall` 13258.
+Signalling: `_handleCallSignal` 13274, `_sendCallSignal`, `MESSAGE_TYPES.CALL_*`.
+Debug: `window.sbCallLog` / `window.__sbCallLog` (module top of the manager).
+UI: `CallUI.jsx` + header buttons (`app.jsx`) + overlay mount in `EnhancedChatInterface`.
+
+## 7. Gaps vs. the target spec
+
+| Requirement | Status | Note |
+|---|---|---|
+| Opus fmtp (FEC/DTX/RED, maxavgbitrate, cbr=0) | ❌ | No SDP munging. |
+| RED via setCodecPreferences | ❌ | No codec prefs. |
+| Audio sender priority/networkPriority/maxBitrate | ❌ | No setParameters. |
+| VP9 SVC (`L3T3_KEY`) + simulcast fallback | ❌ | No `sendEncodings`; uses `addTrack`. |
+| AV1 / H.264 / VP8 fallback order | ❌ | No `getCapabilities` sort. |
+| TWCC/NACK/PLI/FIR/REMB on m-lines | ⚠️ | Whatever the browser emits by default; not enforced/verified. |
+| `NetworkAdaptationController` (getStats loop) | ❌ | None. |
+| Config with sourced constants | ❌ | Only `_config.webrtc` (ICE/privacy). |
+
+## 8. Proposed integration (for confirmation — not yet built)
+
+New **plain-JS** modules under `src/network/webrtc/` (imported by the manager), mapping to the task's requested files:
+
+| Task file | Proposed actual file |
+|---|---|
+| `src/webrtc/config.ts` | `src/network/webrtc/config.js` — all constants + source comments |
+| `src/webrtc/codecs/audio.ts` | `src/network/webrtc/audio.js` — `configureAudioSender` |
+| `src/webrtc/codecs/video.ts` | `src/network/webrtc/video.js` — `configureVideoSender` |
+| `src/webrtc/codecs/sdp.ts` | `src/network/webrtc/sdp.js` — pure SDP munging utils (testable in Node) |
+| `src/webrtc/adaptation/metrics.ts` | `src/network/webrtc/adaptation/metrics.js` — getStats parsing |
+| `src/webrtc/adaptation/controller.ts` | `src/network/webrtc/adaptation/controller.js` — `NetworkAdaptationController` |
+| `src/webrtc/call.ts` | **integrate into `EnhancedSecureWebRTCManager.js`** call methods |
+
+Hook points in the manager:
+- `_acquireLocalMedia` (12999): switch `addTrack` → `addTransceiver('audio'/'video', {direction, sendEncodings})`, then `setCodecPreferences` on the transceivers.
+- `startCall`/`_answerCallOffer`/`_renegotiateCall`: after `createOffer/createAnswer`, run `sdp.js` munging before `setLocalDescription`; after `setRemoteDescription`, call `configureAudioSender`/`configureVideoSender` and start the controller.
+- `_teardownCallMedia`: stop the controller.
+
+**Key compromise to confirm:** the PC is long-lived and shared with the data channel, so we **cannot** follow Step-6's "addTransceiver at PC init" literally. Transceivers are added/negotiated in-band at call start on the existing PC (the audio/video transceivers can be created once on first call and reused via `replaceTrack` thereafter — this also fixes the transceiver-accumulation issue seen earlier).
+
+## 9. Constraints / risks
+
+- **Browser matrix:** `scalabilityMode`/`setCodecPreferences` support varies. Firefox: SVC limited → simulcast fallback mandatory. Safari: VP9 often absent → H.264 fallback; RED support varies. All feature-detected via `getCapabilities`/try-catch.
+- **SDP munging is fragile** across browsers (m-line/payload ordering). Utilities must be defensive and idempotent (spec says "no duplicate lines").
+- **Testing:** SDP utils + controller (mock `getStats`) → plain `node .mjs` unit tests (fits current infra). **Integration test needs a real browser** (two `RTCPeerConnection`s + throttling) — **Playwright is not installed**; that item needs either adding Playwright (a dependency — task says avoid unless necessary) or manual `chrome://webrtc-internals` verification.
+- **In-band renegotiation** must keep the perfect-negotiation guards already in place to avoid glare.
+
+## 10. Open questions before Step 2
+
+1. Confirm JS (not TS) + `src/network/webrtc/` layout in §8.
+2. Confirm the long-lived-PC compromise in §8 (reuse transceivers via `replaceTrack`, negotiate in-band).
+3. Playwright: add it for the integration test, or rely on `chrome://webrtc-internals` manual verification for the "adaptation visible under throttling" criterion?
+4. Commit-per-step + deploy to Fly after each step (as before), GitHub untouched — same as current workflow?
@@ -0,0 +1,160 @@
+# WebRTC call configuration & rationale
+
+All tunables live in `src/network/webrtc/config.js`. This doc explains the values
+and where they come from. Built incrementally per step; **all steps shipped** —
+adaptive audio, SVC video, transport feedback, and runtime bitrate adaptation are
+live in the 1:1 call flow.
+
+## How the pieces attach (why not one `configureAudioSender`)
+
+A single `RTCRtpSender` cannot carry codec ordering or fmtp, so the brief's
+`configureAudioSender` is split across the three WebRTC surfaces at their correct
+lifecycle points (see `EnhancedSecureWebRTCManager` call methods):
+
+| Concern | API surface | When | Code |
+|---|---|---|---|
+| RED-first / Opus-second ordering | `transceiver.setCodecPreferences` | before `createOffer`/`createAnswer` | `applyAudioCodecPreferences` (audio.js) via `_applyAudioCodecPrefs` |
+| Opus FEC/DTX/bitrate | SDP `a=fmtp` munging | after create, before `setLocalDescription` | `applyOpusSettings` (sdp.js) via `_mungeCallSdp` |
+| priority / networkPriority / maxBitrate | `sender.setParameters` | after `setLocalDescription` | `configureAudioSender` (audio.js) via `_applyCallSenderParams` |
+
+Both peers run the same munging, so the negotiated session carries the params.
+
+## Step 2 — Audio (`AUDIO_CONFIG`)
+
+### Opus fmtp (`opusFmtp`)
+| Param | Value | Why |
+|---|---|---|
+| `minptime` | 10 | Smaller packetisation time → lower latency. Opus RFC 7587 §7. |
+| `useinbandfec` | 1 | In-band FEC reconstructs a lost packet from the next one — the main lever for the 15–20% loss target. RFC 6716 §2.1.7. |
+| `usedtx` | 1 | Discontinuous transmission: stop sending in silence, freeing the shared transport for video/FEC. RFC 7587 §3.1.3. |
+| `stereo` | 0 | Mono voice — halves bitrate, no quality loss for speech. |
+| `maxaveragebitrate` | 32000 | Comfortable wideband speech; brief-specified. |
+| `cbr` | 0 | Variable bitrate spends bits only when needed — better quality/bit than CBR for speech. |
+
+### RED (`preferRed: true`)
+RFC 2198 redundant audio: each packet also carries the previous frame's payload,
+so isolated losses recover without retransmission. Enabled only when the browser
+advertises `audio/red` in `RTCRtpSender.getCapabilities('audio')` (Chromium yes;
+Safari/Firefox vary → silently skipped). Ordered **RED first, Opus second** via
+`setCodecPreferences`.
+
+### Sender (`sender`)
+| Param | Value | Why |
+|---|---|---|
+| `maxBitrate` | 40000 bps | Brief value; head-room above 32 kbps Opus for RED redundancy. |
+| `priority` | `high` | Bandwidth arbitration within the PC — audio wins over video. |
+| `networkPriority` | `high` | DSCP hint so audio packets are prioritised on the wire. |
+
+## Step 3 — Video (`VIDEO_CONFIG`)
+
+### Codec preference (`codecPreferenceOrder`)
+`VP9 → AV1 → H.264 → VP8`, applied with `setCodecPreferences` (via
+`applyVideoCodecPreferences`). rtx/red/fec codecs are kept after the media codecs
+so retransmission/FEC still work. VP9/AV1 give SVC; H.264/VP8 are plain.
+
+### Single-encoding SVC (why not multi-rid simulcast)
+This is **1:1 P2P** (one receiver), so a single encoding with **SVC** is the right
+tool: one stream that degrades by spatial/temporal layer. It's applied through
+`sender.setParameters` (`configureVideoSender`) and needs **no** `addTransceiver`/
+rids, so it doesn't disturb the working `addTrack` media path.
+
+| Codec | scalabilityMode | maxBitrate | degradationPreference |
+|---|---|---|---|
+| VP9 | `L3T3_KEY` (3 spatial × 3 temporal, key-aligned) | 1.5 Mbps | `balanced` |
+| AV1 | `L1T3` | 1.2 Mbps | `maintain-framerate` |
+| H.264 / VP8 | none (plain) | 1.5 Mbps | `balanced` |
+
+`networkPriority: 'medium'` — below audio's `high`. If a browser rejects the SVC
+mode (Firefox/Safari gaps), `configureVideoSender` retries with a plain encoding.
+
+### Initialization (Step 6) + simulcast (Step 3) — what shipped, and why
+
+**Attempted then reverted:** an explicit `addTransceiver({sendEncodings})` path
+(single-encoding SVC / multi-rid simulcast) was implemented but **broke media on
+real devices** during testing:
+- on the **answerer**, reusing the transceiver created by `setRemoteDescription`
+  rejected the SVC `setParameters` (`"parameters are not valid"`), and
+- on **role-reversed / repeat calls**, the reused transceiver **directions
+  desynced** — the call connected (timer ran, `ontrack` fired) but no audio/video
+  actually flowed.
+
+**What ships instead:** media is attached with **`addTrack`** (reused across calls
+via `replaceTrack`), letting the browser manage transceiver direction implicitly —
+this is what keeps audio/video flowing across reversed and repeat calls. Video uses
+**single-encoding SVC** (VP9 `L3T3_KEY` / AV1 `L1T3`) applied via
+`configureVideoSender`→`setParameters`, plus `setCodecPreferences` (VP9→AV1→H264→VP8).
+For 1:1 P2P a single SVC stream is the right tool anyway — it degrades by
+spatial/temporal layer, which is what the "video degrades gracefully" requirement
+needs. `configureVideoSender` falls back to a plain encoding if a scalabilityMode
+is rejected.
+
+**Multi-rid simulcast** (`buildVideoSendEncodings`, `VIDEO_CONFIG.*.simulcast`) is
+kept as **tested, exported primitives** for a future SFU/group-call path, but is
+**not wired into the 1:1 call flow** — it needs `addTransceiver({sendEncodings})`,
+which requires the answerer-direction / setParameters issues above to be solved
+first (ideally with a Playwright two-PC test rig, which isn't set up). The
+adaptation controller is already simulcast-aware (`_applyToVideoSender` throttles
+only the top layer) for when that lands.
+
+## Step 4 — Transport (`TRANSPORT_CONFIG`)
+
+Ensures the RTCP feedback / header extensions are present on the call m-lines
+(most browsers already emit them, so this is an idempotent safety net):
+
+| m-line | rtcp-fb | header ext |
+|---|---|---|
+| video | `transport-cc`, `nack`, `nack pli`, `ccm fir`, `goog-remb` | TWCC (`…transport-wide-cc…`) |
+| audio | `transport-cc`, `nack` | TWCC |
+
+Implemented as pure, idempotent SDP utils (`ensureRtcpFb`, `ensureExtmap`,
+`applyTransport`) — added only when missing, never duplicated, applied only to
+primary codecs (rtx/red/fec skipped). `transport-cc` (TWCC) is what feeds the
+bandwidth estimator the Step-5 controller reads.
+
+**Safety:** munged local SDP is set via `_setLocalMunged` with progressive
+fallback — full munge → Opus-only → raw — so a browser rejecting an injected
+line can never break the call.
+
+## Step 5 — Adaptation + quality indicator (`ADAPTATION_CONFIG`)
+
+`NetworkAdaptationController` reads `pc.getStats()` every 1 s and reacts:
+
+| Condition | Action | Note |
+|---|---|---|
+| loss > 10% **or** RTT > 300 ms | video `maxBitrate` −20% (floor 100 kbps) | audio never touched |
+| loss < 3% **and** RTT < 150 ms, 5 ticks | video `maxBitrate` +10% (up to ceiling) | gradual recovery |
+| `qualityLimitationReason === 'cpu'` | `scaleResolutionDownBy` ×1.5 | bitrate unchanged |
+
+All changes go through `sender.setParameters` — **no renegotiation, no track
+restart**. **Audio is never throttled** by the controller, so speech stays intact
+regardless of loss (stronger than the brief's "≥25%" guard). The pure decision
+(`decideAdaptation`) and stats parsing (`summarizeStats`) are unit-tested with
+mock getStats.
+
+### Connection-quality indicator (user-facing)
+The same `getStats` sample yields a coarse label (`qualityFromMetrics`) surfaced in
+`callState.quality` and rendered in the call UI as signal bars + text:
+
+| Label | Loss / RTT | Colour |
+|---|---|---|
+| Excellent | <3% and <150 ms | green |
+| Good | <7% and <250 ms | green |
+| Fair | <15% and <400 ms | yellow |
+| Weak | otherwise | red |
+
+Shown in the voice overlay, the video top bar, and (compact bars) the minimized
+widget. Hidden until the first sample has data.
+
+## Verification (all steps)
+- Unit: `npm test` (SDP, video codecs, adaptation) — all green.
+- Manual (`chrome://webrtc-internals`, per criteria): throttle the link (DevTools
+  Network / OS) → outbound video `targetBitrate` steps down within ~1–2 s and
+  recovers when the link clears; audio bitrate holds; the in-call indicator moves
+  Excellent → Fair → Weak.
+- Audio: in a call, open `chrome://webrtc-internals` → audio outbound-rtp should
+  show Opus with the fmtp above; on Chromium the codec is `red`/`opus`. Under
+  packet loss, `useinbandfec` keeps audio intelligible.
+
+> The temporary `[Call]` / `window.__sbCallLog` diagnostic logger used during
+> bring-up has been removed for production; the manager's `_secureLog` handles
+> runtime logging and stays silent in production builds (`DEBUG_MODE = false`).