Files
ai-podcast/tests/test_proper_nouns.py
T
lukeandClaude Opus 5 2901e5f4fb Stop Whisper misspelling the intern's name as Devin
The intern reached 21 of 58 published transcripts as "Devin". The obvious fix —
seed his name in the Whisper initial prompt — only works for one of the two
transcription paths:

- backend/services/transcription.py (live show) uses mlx_whisper and does take
  an initial_prompt. Added Devon there. This also picks up the Big Bend prompt
  rewrite from the relocation work, which was sitting uncommitted.
- publish_episode.py, which actually produces the published transcripts, uses
  LightningWhisperMLX, whose transcribe() signature is (audio_path, language).
  It accepts no initial_prompt at all, so there is nothing to seed.

So the published path gets a deterministic correction pass instead:
fix_proper_nouns() rewrites known mishearings after transcription, preserving
casing (Devin/DEVIN/devin -> Devon/DEVON/devon) and matching whole words only,
so "Devinshire" is left alone. Verified against the real episode 58 transcript:
24 occurrences to 0, output byte-identical to the manual relabel in bf1afef.

Swapping the publish path to mlx_whisper would allow a real prompt, but that
changes the transcription engine for every episode and is a bigger call than
this warrants.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-14 04:20:22 -05:00

53 lines
1.8 KiB
Python

"""Whisper mishears recurring show names; correct them after transcription.
publish_episode.py transcribes with LightningWhisperMLX, whose transcribe()
signature is (audio_path, language) — it accepts no initial_prompt, so there is
no way to condition it on proper nouns the way make_clips.py does with
mlx_whisper. The intern came out as "Devin" in 21 of 58 published transcripts
because of this.
A deterministic pass over the finished text fixes the known names without
swapping the transcription engine.
"""
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from publish_episode import fix_proper_nouns
def test_corrects_the_intern_name_in_all_casings():
assert fix_proper_nouns("Devin, where's my coffee?") == "Devon, where's my coffee?"
assert fix_proper_nouns("DEVIN: Hey Luke.") == "DEVON: Hey Luke."
assert fix_proper_nouns("devin said so") == "devon said so"
def test_leaves_correct_spelling_alone():
text = "Devon is the intern. DEVON: Hi."
assert fix_proper_nouns(text) == text
def test_only_matches_whole_words():
"""Must not corrupt a longer word that happens to contain the name."""
assert fix_proper_nouns("Devinshire") == "Devinshire"
assert fix_proper_nouns("mcdevins") == "mcdevins"
def test_preserves_surrounding_text_exactly():
src = "LUKE: Alright. Let's check in with Devin and see how he's doing.\n\nDEVIN: Hey!"
out = fix_proper_nouns(src)
assert out == "LUKE: Alright. Let's check in with Devon and see how he's doing.\n\nDEVON: Hey!"
assert len(out) == len(src)
def test_empty_and_none_safe():
assert fix_proper_nouns("") == ""
assert fix_proper_nouns(None) == ""
def test_word_count_is_never_changed():
src = "Devin talked to Devin about Devin. " * 20
assert len(fix_proper_nouns(src).split()) == len(src.split())