Play the audio and follow the subtitles. Each word is colored by the speaker that method assigned; a red underline means it disagrees with ground truth.
Source: Harper Valley Bank (CC BY 4.0) — no patient audio.
What each method knows:AWS and W+ECAPA self-enrolled are fully blind — profiles
come from the recording's own clusters, with no outside reference audio. W+ECAPA enrolled and FT v2
were given about 20 s of reference audio from each person's clean channel — available for care partners
in the archive, rarely for patients. Compare the first two for a fair test, the last two to see what enrollment adds.
Scores
Subtitles
Extraction quality — pretrained vs our fine-tunes
A different question, and the one extraction is actually built for: on a natural call, how cleanly does the model pull out one person?
Three held-out Harper Valley calls, agent enrolled. Scores are SI-SNR in dB while the target speaks alone (higher is better),
and the other speaker's residue (lower is better). The pretrained model was the best of three off-the-shelf options; the fine-tunes are ours.
FT v2 — extracted audio
The other methods only label; they never split the audio. FT v2 actually extracts it — this is the per-person output.
Transcript quality — per-speaker WER
Of each speaker's reference transcript, how much did each method get right? WER, lower is better; best per row highlighted.
MOSS and AWS are single-pass systems; W+ECAPA splits one Whisper pass; DiCoW re-runs conditioned ASR per speaker; FT v2 extracts audio then transcribes.
case
speaker
MOSS
AWS
W+ECAPA
DiCoW
FT v2
two men (1%)
speaker 1
%0.0
%4.8
%4.8
%27.4
%46.8
two men (1%)
speaker 2
%0.0
%14.0
%14.0
%36.0
%58.0
two women (5%)
speaker 1
%13.1
%36.9
%33.1
%32.3
%33.8
two women (5%)
speaker 2
%22.4
%48.0
%41.8
%27.6
%75.5
dense overlap (47%)
speaker 1
%81.0
%98.4
%96.8
%54.0
%55.6
dense overlap (47%)
speaker 2
%9.5
%9.5
%11.9
%4.8
%7.1
three speakers (55%)
speaker 1
%88.3
%100.0
%100.0
%95.0
%71.7
three speakers (55%)
speaker 2
%55.0
%100.0
%95.0
%37.5
%82.5
three speakers (55%)
speaker 3
%42.9
%38.1
%52.4
%47.6
%35.7
Two regimes, two winners. On natural calls MOSS is the clear leader (0–22% WER).
On heavily overlapped speech, DiCoW — Whisper conditioned on the diarization mask — is the only method that writes the masked
speaker as text (54% vs 97–98% for every labeling approach), matching audio extraction while beating it on the dominant speaker.
DiCoW's catch: it needs a diarization input (pyannote, the slowest stage) and its released code pins old library versions.
Speed
Measured end to end on the 107 s two-men call, one laptop CPU, no GPU. Model loading (about 8 s total) excluded.
stage
time
vs real time
Whisper large-v3-turbo (words)
27.5 s
3.9x
pyannote (clustering)
96.3 s
1.1x
ECAPA (423 sliding windows)
9.3 s
11.5x
SpeakerBeam FT v2 (one speaker)
25.5 s
4.2x
full pipeline
107 s call
vs real time
W+ECAPA enrolled (no pyannote)
36.8 s
2.9x
MOSS end-to-end (one pass, no pyannote, no Whisper)
93 s
1.2x
DiCoW (pyannote 96 s + conditioned ASR 80 s)
176 s
0.6x
W+ECAPA self-enrolled
133.1 s
0.8x
plus FT v2 on both speakers
184.1 s
0.6x
Pipeline totals for 107 s of audio:
W+ECAPA enrolled 36.8 s (2.9x) · W+ECAPA self-enrolled 133.1 s (0.8x) · plus FT v2 on both speakers 184.1 s (0.6x).
pyannote is the bottleneck — it costs more than everything else combined. A CP voice profile removes it from the pipeline,
which is why enrollment is worth about 3.6x in speed on top of what it does for accuracy.