lockbitchat
3212138a0d
No protocol or message-protection changes. The version in the application header was a literal and had fallen behind, showing v5.6.0 while running 5.7.1. It now comes from package.json, and a test fails if a hard-coded one reappears or if meta.json, the README badge, the changelog and the docs disagree about the release. Documentation reorganised so that everything technical lives in doc/ with an index, and the root keeps only what belongs there by convention: README, SECURITY, CHANGELOG and LICENSE. - SECURITY.md rewritten. It listed a supported release line three major versions out of date and made claims the software does not make. It now states what is guaranteed, what is not, and how to report a problem. - SECURITY_DISCLAIMER.md and RESPONSIBLE_USE.md merged into doc/USE-POLICY.md, which says what the software cannot protect against rather than listing generic advice. - doc/SECURITY-ARCHITECTURE.md renamed to doc/ARCHITECTURE.md and rewritten around the session lifecycle, what verification gates, and how recovery works. - doc/CRYPTOGRAPHY.md rewritten: key schedule, the Double Ratchet, framing, and memory handling, with values taken from the source rather than restated. - doc/CONFIGURATION.md rewritten with the real file-type policy, ICE and TURN guidance, and the deployment caching rules that matter. - docs/webrtc-config.md moved to doc/CALLS.md and rewritten; the obsolete docs/webrtc-audit.md, a working document full of stale line numbers, removed along with the docs/ directory. - doc/CONTRIBUTING.md records what the recent regressions taught us about writing tests that can actually fail. - doc/README.md added as an index. Internal security review notes are excluded from the repository via .gitignore. Those describe attack paths against specific releases in enough detail to reproduce them, which is useful privately and harmful in public while users are still updating.
171 lines
7.5 KiB
Markdown
171 lines
7.5 KiB
Markdown
# Voice and video calls
|
|
|
|
Calls run over the same peer connection as the chat. Media is added to the
|
|
existing connection and renegotiated onto it, and the call SDP is exchanged over
|
|
the encrypted data channel rather than through any signalling service. Media
|
|
therefore inherits the session's verification: the DTLS-SRTP fingerprints
|
|
negotiated for the media were themselves carried over an authenticated channel.
|
|
|
|
All tunable values live in `src/network/webrtc/config.js`. This document explains
|
|
where they come from, because the numbers are otherwise indistinguishable from
|
|
arbitrary choices.
|
|
|
|
## Where the settings attach
|
|
|
|
A single `RTCRtpSender` cannot express codec ordering or fmtp parameters, so
|
|
configuring a sender is spread across three WebRTC surfaces, each at the point in
|
|
the lifecycle where it works:
|
|
|
|
| Concern | Surface | When | Implementation |
|
|
| --- | --- | --- | --- |
|
|
| Codec ordering (RED before Opus) | `transceiver.setCodecPreferences` | before creating the offer or answer | `applyAudioCodecPreferences` in `audio.js` |
|
|
| Opus FEC, DTX, bitrate | SDP `a=fmtp` rewriting | after create, before `setLocalDescription` | `applyOpusSettings` in `sdp.js` |
|
|
| Priority and maximum bitrate | `sender.setParameters` | after `setLocalDescription` | `configureAudioSender` in `audio.js` |
|
|
|
|
Both peers apply the same rewriting, so the negotiated session carries the
|
|
parameters regardless of who called.
|
|
|
|
## Audio
|
|
|
|
Speech has to stay intelligible on a bad link, and that goal drives every value
|
|
below.
|
|
|
|
### Opus parameters
|
|
|
|
| Parameter | Value | Reason |
|
|
| --- | --- | --- |
|
|
| `minptime` | 10 | Smaller packetisation interval, lower latency (RFC 7587, section 7) |
|
|
| `useinbandfec` | 1 | In-band forward error correction reconstructs a lost packet from the next one. This is the main lever for staying intelligible at 15 to 20 percent loss (RFC 6716, section 2.1.7) |
|
|
| `usedtx` | 1 | Discontinuous transmission stops sending during silence, leaving the transport free for video and FEC (RFC 7587, section 3.1.3) |
|
|
| `stereo` | 0 | Mono halves the bitrate with no loss for speech |
|
|
| `maxaveragebitrate` | 32000 | Comfortable wideband speech |
|
|
| `cbr` | 0 | Variable bitrate spends bits only when needed |
|
|
|
|
### Redundancy
|
|
|
|
RED (RFC 2198) carries the previous frame's payload alongside each packet, so
|
|
isolated losses recover without retransmission. It is enabled only when the
|
|
browser advertises `audio/red` in `RTCRtpSender.getCapabilities('audio')`, which
|
|
Chromium does and others vary on; when absent it is skipped silently. RED is
|
|
ordered before Opus in codec preferences.
|
|
|
|
### Sender parameters
|
|
|
|
| Parameter | Value | Reason |
|
|
| --- | --- | --- |
|
|
| `maxBitrate` | 40000 bps | Headroom above the 32 kbps Opus target for RED redundancy |
|
|
| `priority` | `high` | Audio wins bandwidth arbitration against video within the connection |
|
|
| `networkPriority` | `high` | DSCP hint so audio is prioritised on the wire |
|
|
|
|
Audio is never throttled by the adaptation controller. Under sustained loss the
|
|
video degrades and speech continues.
|
|
|
|
## Video
|
|
|
|
Codec preference order is VP9, AV1, H.264, VP8, applied through
|
|
`setCodecPreferences`. Retransmission and FEC codecs are kept after the media
|
|
codecs so they still function. VP9 and AV1 provide scalable coding; H.264 and VP8
|
|
do not.
|
|
|
|
### Scalable coding rather than simulcast
|
|
|
|
This is a one-to-one connection with a single receiver, so one encoding with SVC
|
|
is the right tool: a single stream that degrades by spatial or temporal layer. It
|
|
is applied through `sender.setParameters` and needs no `addTransceiver` or rid
|
|
configuration, which keeps it away from the media path that actually works.
|
|
|
|
| Codec | scalabilityMode | maxBitrate | degradationPreference |
|
|
| --- | --- | --- | --- |
|
|
| VP9 | `L3T3_KEY` (3 spatial, 3 temporal, key-aligned) | 1.5 Mbps | `balanced` |
|
|
| AV1 | `L1T3` | 1.2 Mbps | `maintain-framerate` |
|
|
| H.264, VP8 | none | 1.5 Mbps | `balanced` |
|
|
|
|
`networkPriority` is `medium`, below audio. If a browser rejects the scalability
|
|
mode, which Firefox and Safari do in places, `configureVideoSender` retries with
|
|
a plain encoding.
|
|
|
|
### Why media is attached with addTrack
|
|
|
|
An explicit `addTransceiver({ sendEncodings })` path was built and then removed,
|
|
because it broke media on real devices in two ways. On the answering side,
|
|
reusing the transceiver created by `setRemoteDescription` rejected the SVC
|
|
parameters outright. On repeat and role-reversed calls, the reused transceiver
|
|
directions desynchronised: the call connected and `ontrack` fired, but no media
|
|
flowed.
|
|
|
|
What ships instead attaches media with `addTrack`, reused across calls through
|
|
`replaceTrack`, and lets the browser manage transceiver direction. This is what
|
|
keeps audio and video flowing across reversed and repeated calls.
|
|
|
|
Multi-rid simulcast primitives (`buildVideoSendEncodings`) are kept and tested for
|
|
a future group-call path, but are not wired into the one-to-one flow. They need
|
|
`addTransceiver({ sendEncodings })`, which requires the problems above to be
|
|
solved first, ideally with a two-connection browser test rig that is not set up.
|
|
The adaptation controller is already simulcast-aware for when that lands.
|
|
|
|
## Transport feedback
|
|
|
|
The call m-lines need RTCP feedback and header extensions present. Most browsers
|
|
emit them already, so this is an idempotent safety net:
|
|
|
|
| Media | RTCP feedback | Header extension |
|
|
| --- | --- | --- |
|
|
| Video | `transport-cc`, `nack`, `nack pli`, `ccm fir`, `goog-remb` | transport-wide congestion control |
|
|
| Audio | `transport-cc`, `nack` | transport-wide congestion control |
|
|
|
|
These are added only when missing, never duplicated, and applied only to primary
|
|
codecs. Transport-wide congestion control is what feeds the bandwidth estimator
|
|
that adaptation reads.
|
|
|
|
Rewritten local SDP is applied with progressive fallback: full rewrite, then Opus
|
|
only, then raw. A browser that rejects an injected line cannot break the call.
|
|
|
|
## Adaptation
|
|
|
|
`NetworkAdaptationController` samples `pc.getStats()` every second and reacts:
|
|
|
|
| Condition | Action |
|
|
| --- | --- |
|
|
| Loss above 10 percent, or round trip above 300 ms | Reduce video `maxBitrate` by 20 percent, floor 100 kbps |
|
|
| Loss below 3 percent and round trip below 150 ms, sustained 5 samples | Raise video `maxBitrate` by 10 percent, up to the ceiling |
|
|
| `qualityLimitationReason` is `cpu` | Scale resolution down by 1.5, bitrate unchanged |
|
|
|
|
Every change goes through `sender.setParameters`. There is no renegotiation and
|
|
no track restart, so adaptation is invisible to the call. Audio is never touched.
|
|
|
|
The decision function and the stats parsing are pure and unit-tested against
|
|
recorded `getStats` output.
|
|
|
|
### Quality indicator
|
|
|
|
The same sample produces a coarse label shown in the call interface:
|
|
|
|
| Label | Condition |
|
|
| --- | --- |
|
|
| Excellent | Loss below 3 percent and round trip below 150 ms |
|
|
| Good | Loss below 7 percent and round trip below 250 ms |
|
|
| Fair | Loss below 15 percent and round trip below 400 ms |
|
|
| Weak | Anything else |
|
|
|
|
It appears in the voice overlay, the video top bar, and as compact bars in the
|
|
minimized widget, and stays hidden until the first sample has data.
|
|
|
|
## Verifying a change
|
|
|
|
Unit tests cover SDP rewriting, video codec selection and the adaptation
|
|
decision:
|
|
|
|
```bash
|
|
npm test
|
|
```
|
|
|
|
For anything touching media in practice, open `chrome://webrtc-internals` during
|
|
a call and check that:
|
|
|
|
- outbound audio shows Opus with the fmtp parameters above, appearing as `red`
|
|
and `opus` on Chromium
|
|
- throttling the link steps outbound video `targetBitrate` down within a second
|
|
or two and recovers when the link clears
|
|
- audio bitrate holds steady while video adapts
|
|
- the in-call indicator moves through Fair and Weak as the link degrades
|