Play the audio and follow the subtitles. Each word is colored by the speaker that method assigned; a red underline means it disagrees with ground truth.
Source: Harper Valley Bank (CC BY 4.0) — no patient audio.
What each method knows:AWS and W+ECAPA self-enrolled are fully blind — profiles
come from the recording's own clusters, with no outside reference audio. W+ECAPA enrolled and FT v2
were given about 20 s of reference audio from each person's clean channel — available for care partners
in the archive, rarely for patients. Compare the first two for a fair test, the last two to see what enrollment adds.
Scores
Subtitles
Extraction quality — pretrained vs our fine-tunes
A different question, and the one extraction is actually built for: on a natural call, how cleanly does the model pull out one person?
Three held-out Harper Valley calls, agent enrolled. Scores are SI-SNR in dB while the target speaks alone (higher is better),
and the other speaker's residue (lower is better). The pretrained model was the best of three off-the-shelf options; the fine-tunes are ours.
FT v2 — extracted audio
The other methods only label; they never split the audio. FT v2 actually extracts it — this is the per-person output.
Transcript quality — per-speaker accuracy
Of each speaker's reference transcript, how much did each method get right? Accuracy (100 − WER), higher is better; best per row highlighted.
MOSS and AWS are single-pass systems; W+ECAPA splits one Whisper pass; DiCoW re-runs conditioned ASR per speaker; FT v2 extracts audio then transcribes.
case
speaker
MOSS
AWS
W+ECAPA
DiCoW
FT v2
two men (1%)
speaker 1
%100.0
%95.2
%95.2
%72.6
%53.2
two men (1%)
speaker 2
%100.0
%86.0
%86.0
%64.0
%42.0
two women (5%)
speaker 1
%86.9
%63.1
%66.9
%67.7
%66.2
two women (5%)
speaker 2
%77.6
%52.0
%58.2
%72.4
%24.5
dense overlap (47%)
speaker 1
%19.0
%1.6
%3.2
%46.0
%44.4
dense overlap (47%)
speaker 2
%90.5
%90.5
%88.1
%95.2
%92.9
three speakers (55%)
speaker 1
%11.7
%0.0
%0.0
%5.0
%28.3
three speakers (55%)
speaker 2
%45.0
%0.0
%5.0
%62.5
%17.5
three speakers (55%)
speaker 3
%57.1
%61.9
%47.6
%52.4
%64.3
Two regimes, two winners. On natural calls MOSS is the clear leader (78–100% accuracy).
On heavily overlapped speech, DiCoW — Whisper conditioned on the diarization mask — is the only method that writes the masked
speaker as text (46% accuracy vs 2–3% for every labeling approach), matching audio extraction while beating it on the dominant speaker.
DiCoW's catch: it needs a diarization input (pyannote, the slowest stage) and its released code pins old library versions.
Speed
Measured end to end on the 107 s two-men call, one laptop CPU, no GPU — except the MLX row, which is the same laptop's Apple GPU. Model loading (about 8 s total) excluded.
stage
time
vs real time
Whisper large-v3-turbo (words)
27.5 s
3.9x
pyannote (clustering)
96.3 s
1.1x
ECAPA (423 sliding windows)
9.3 s
11.5x
SpeakerBeam FT v2 (one speaker)
25.5 s
4.2x
MOSS (no stages — one pass replaces all of the above)
93 s CPU · 8.2 s MLX
1.2x · 13.1x
full pipeline
107 s call
vs real time
W+ECAPA enrolled (no pyannote)
36.8 s
2.9x
MOSS end-to-end (one pass, no pyannote, no Whisper)
93 s
1.2x
MOSS 8-bit MLX (same weights, Apple GPU, single pass)
8.2 s
13.1x
DiCoW (pyannote 96 s + conditioned ASR 80 s)
176 s
0.6x
W+ECAPA self-enrolled
133.1 s
0.8x
plus FT v2 on both speakers
184.1 s
0.6x
Pipeline totals for 107 s of audio:
W+ECAPA enrolled 36.8 s (2.9x) · W+ECAPA self-enrolled 133.1 s (0.8x) · plus FT v2 on both speakers 184.1 s (0.6x).
pyannote is the bottleneck — it costs more than everything else combined. A CP voice profile removes it from the pipeline,
which is why enrollment is worth about 3.6x in speed on top of what it does for accuracy.
The 8-bit MLX build changes the picture entirely: the same MOSS pass at 13x real time, transcript and speaker labels
identical to the full-precision output on our test files — fast enough that one small GPU covers production volume.