Commit Graph
181 Commits
Author SHA1 Message Date
lukeandClaude Opus 4.6 8a20e679f6 Fix caller name mismatch after mid-session theme change
Setting a show theme after the caller lineup was loaded caused every
caller button to keep its pre-theme label while the LLM dialog used
the post-theme bg names — host says "Hi Phil", caller replies "it's
Cody, actually". Three underlying bugs, all fixed here:

- _regenerate_backgrounds_for_keys ignored its keys parameter and
  replaced the entire caller_backgrounds dict, clobbering used slots
  along with unused ones. Now only the listed slots are touched.

- set_show_theme's used_keys detection compared record.caller_name
  (the slim bg name) against CALLER_BASES[k]["name"] (the randomized
  fallback name) — two different namespaces that never match, so
  every slot was flagged unused. Now it matches against
  session.caller_backgrounds[k]["name"].

- set_show_theme fired the regeneration as a detached asyncio task,
  so the POST returned before bg was consistent. Even if the frontend
  did reload /api/callers, it would race the regen. Now awaited.

Frontend setShowTheme/clearShowTheme now call loadCallers() after
the theme POST resolves so the button list actually refreshes, with
a "Regenerating..." button state during the ~30-60s wait.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 02:00:31 -06:00
lukeandClaude Opus 4.6 35ff78a922 Merge branch 'feature/caller-redesign' into main
Brings in the Phase 5B caller-generation rewrite:
- Slim two-stage pipeline: Sonnet 4.6 batch identity pregen at session
  start, Haiku 4.5 live dialog per turn. Deletes CallerBackground
  dataclass, shape/style/voice-matching, per-model routing, preflight UI.
- New caller_gen.py (slim prompt builder) and regulars_v2.py (Obsidian
  lore loader for named recurring callers).
- Reworked caller buttons and info panel around the slim background:
  identity, situation, signature, secret want. Removes shape badges,
  energy dots, emotion info-badges.
- Inworld TTS: emotional_register -> (temperature, speed_adjust) mapping
  via _emotional_register_to_params(). applyTextNormalization ON.
- Silas identity/voice leak fix, avatar gender, pre-warm batch gen.
- Archives old regulars to data/regulars.archived.json, adds 10 sample
  caller transcripts under docs/samples/.

Post-merge fixes applied during resolution:
- intern.py: kept main's richer new_show() (NEW SHOW history marker,
  trim, _save()) and _track_suggestion(); removed the branch's
  duplicate minimal new_show() stub.
- tts.py: added 5 voice speed overrides (Graham, Malcolm, Victoria,
  Loretta, Marlene) that main had tuned locally. Evelyn deliberately
  NOT added to VOICE_PROFILES — she is in BLACKLISTED_VOICES for
  unnatural prosody.
- Main's cost dashboard polish, LLM per-model params, and Devon show
  context memory from commits c087c03..61b3cba carried through cleanly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 23:43:56 -06:00
lukeandClaude Opus 4.6 61b3cba412 Publishing, website, reaper, and doc updates
Publish script, clip maker, website worker + data, reaper lua helpers,
audio settings, and CLAUDE.md doc reorg.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 23:39:57 -06:00
lukeandClaude Opus 4.6 44ac569fec Devon: track show transitions and avoid repeating suggestions
Adds new_show() that inserts a NEW SHOW marker into Devon's history
so he won't reference previous episodes' callers or topics, trims
older history while keeping some long-term memory, and clears
research_cache + pending suggestions. Buffered suggestions are now
tracked in history via _track_suggestion() so Devon won't suggest
the same thing twice. Devon history scoped to current show on ask.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 23:39:52 -06:00
lukeandClaude Opus 4.6 54ad04ae09 LLM: per-model params for caller_dialog + longer timeouts
Different models need different tuning for natural conversation:
Qwen gets high frequency penalty to fight repetition loops, Llama
gets warmer temp to reduce terseness, Grok/Mistral/DeepSeek/Kimi get
slightly warmer than Sonnet defaults. Bumps base httpx timeout from
10s to 30s and fallback per-call timeout from 8s to 20s.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 23:39:45 -06:00
lukeandClaude Opus 4.6 c087c0362a Cost dashboard: TTS providers endpoint, header link, polish
Adds /api/costs/tts endpoint backed by get_tts_providers() in cost_db,
surfaces a Costs link in the control panel header, and small polish
to the costs page HTML/JS.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 23:39:40 -06:00
lukeandClaude Opus 4.6 d0bb50fe12 Fix Silas identity/voice leak, avatar gender, and add pre-warm batch gen
session.caller was reading name/voice from CALLER_BASES randomized defaults
instead of the slim bg dict — causing Silas to appear as "Earl" with wrong
voice. Now prefers bg data when populated.

Also: avatar endpoint infers gender from voice pool, /api/callers returns bg
name, regulars get canonical voice force-locked, batch gen splits into two
parallel calls with pre-warm so subsequent resets are instant.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 19:08:46 -06:00
luke 4e28f3fb84 Tune Inworld TTS: temperature, text normalization, emotional register mapping
Changes to backend/services/tts.py:
- Add temperature: 0.9 and applyTextNormalization: "ON" to Inworld payload
  (text normalization auto-speaks "$5,432", "Dr.", phone numbers, dates, etc.)
- Add _emotional_register_to_params() mapping caller's emotional_register
  to (temperature, speed_adjust) across 5 families: sadness/grief,
  anger/aggression, manic/excited, nervous/earnest, gruff/restrained.
- generate_speech_inworld() and generate_speech() now accept optional
  emotional_register kwarg; all providers take it via the dispatch lambdas
  (only inworld uses it; others ignore)
- Clamp speed to 0.5-1.5 after both emotional-register and per-text adjustments

In backend/main.py: plumb emotional_register through the two caller-dialog
TTS calls (auto-respond and ai-respond) by reading from the slim caller
background dict. Devon and cohost/announcer calls pass empty string and
hit the default (0.9, 0.0) branch — no behavior change.
2026-04-05 14:54:15 -06:00
luke 9e37fbf124 Phase 5B commit 6: delete CallerBackground dataclass, relationship_context, update docs
Final cleanup pass for the caller generation redesign:
- Delete CallerBackground dataclass + all isinstance() checks
- Delete orphaned _build_relationship_context and session.relationship_context
- Delete Session.get_caller_model (caller_dialog category routes to haiku-4.5)
- Delete dead _get_show_energy (CallRecord no longer tracks energy_level)
- Delete unused tone_streak field
- Drop topic_category/emotional_state/energy_level from CallRecord
- Simplify caller property, _find_thematic_match, enrich, promotion paths to slim dict
- Drop broken emotional_state=/energy_level= kwargs from generate_speech calls
- _load_checkpoint drops pre-slim schema backgrounds; startup re-pregens if empty
- Rewrite CLAUDE.md "Caller Generation System" section for the new two-stage architecture

35 tests passing. backend/main.py: 4981 -> 4810 lines.
2026-04-05 13:56:08 -06:00
luke 83c7f441a8 Phase 5B commit 5: delete template generation, content pools, and topic history
Template-based caller generation has been replaced by the batch sonnet-4.6
path (caller_gen.generate_batch). All the template-era content now goes:

Template functions (~700 lines):
- _generate_returning_caller_background{_sync}
- _generate_pool_weights, _is_spicy, _is_absurd, _filter_used
- _pick_unique_reason
- generate_caller_background (the big template fallback)
- _generate_caller_background_llm
- pick_location, _get_time_context, _get_moon_phase, _get_seasonal_context

Content pools (~4600 lines):
- PROBLEMS, STORIES, ADVICE, GOSSIP, HOT_TAKES, CELEBRATIONS, WEIRD,
  TOPIC_CALLIN, TV_TONIGHT, PROBLEM_FILLS
- INTERESTS, QUIRKS, PEOPLE_MALE, PEOPLE_FEMALE, RELATIONSHIP_STATUS,
  CALLING_FROM, MEMORIES, STRONG_OPINIONS, VERBAL_TICS
- JOBS_MALE, JOBS_FEMALE, LOCATIONS_LOCAL, LOCATIONS_OUT_OF_STATE
- CALLER_STYLES, CALLER_STYLE_KEYS

Topic history (~50 lines):
- TOPIC_HISTORY_FILE, _topic_history, _load_topic_history,
  _save_topic_to_history

Orphaned helpers:
- SHAPE_DIRECTIVES, _get_pacing_block, _get_speech_block

Rewrite _regenerate_backgrounds_for_keys to re-run _pregenerate_backgrounds
so the batch path handles theme-change regen too. Rewrite
Session.get_caller_background to be a pure getter — backgrounds are
populated by _pregenerate_backgrounds at session start, no lazy fallback.

Drop session.pool_weights field and its checkpoint save/load.
2026-04-05 13:38:54 -06:00
luke 9982f65742 Phase 5B commit 4: delete style/shape/voice-matching/model-routing systems
Backend cleanup:
- Delete STYLE_SPEED_MODIFIERS, STYLE_PHONE_QUALITY constants
- Delete _normalize_style_key, _match_voices_to_styles,
  get_style_speed_modifier, get_style_phone_quality functions
- Delete _pick_caller_style, _HEAVY_STYLES, _LIGHT_STYLES,
  _EVASIVE_STYLES (only called by template path)
- Delete _is_informal_style + _INFORMAL_STYLES
- Remove orphaned calls to deleted _assign_call_shape
- Drop shape parameter from _pick_response_budget
- Simplify _assess_call_quality (no more shape/style/pool_name)
- Delete Session fields: caller_styles, caller_shapes,
  caller_model_strategy/pool/map/fallback/models/_cycle_idx
- Remove those fields from Session.reset, checkpoint save/load,
  and session.caller property
- Drop style/shape lookups in chat callsites and promotion gate
- Update config.py caller_dialog model stale comment
2026-04-05 13:17:37 -06:00
luke ab811bda91 Delete caller-model routing + preflight admin UI
Backend (~273 lines):
- Removed GET/POST /api/caller-models endpoints (strategy/pool/map/fallback config)
- Removed POST /api/caller-models/{caller_key} override endpoint
- Removed GET /api/show/preflight diagnostics endpoint

Frontend:
- HTML: dropped Preflight header button, Caller Models settings panel,
  Preflight modal, and caller-model-badge/override from info panel.
- JS (~328 lines): removed loadCallerModels, saveCallerModels,
  updateCallerModelUI/Badges, overrideCallerModel, showCallerModelBadge,
  populateCallerModelOverride, runPreflight + render helpers,
  MODEL_ABBREVS, CALLER_STYLES, and all their listeners and init calls.
- CSS (~184 lines): removed .cm-*, .preflight-*, .caller-model-*,
  .model-tag, .info-badge.model, .caller-model-override rules.

Session fields (caller_model_strategy/pool/map/fallback/models,
caller_styles, caller_shapes) still present — they wire into shape
picking and checkpoint restore and get deleted in the next commits.
2026-04-05 12:42:59 -06:00
luke 13d613df5f Rework caller buttons and info panel for slim backgrounds
Backend:
- /api/callers now returns slim fields: identity, situation, signature,
  secret_want, voice (was: energy_level, emotional_state,
  communication_style, signature_detail, situation_summary, pool_name,
  call_shape).
- /api/call/{caller_key} caller_info swapped to slim fields too.

Frontend:
- Caller buttons stripped to name + shortcut + returning star. No more
  energy-dots or shape-badges — those systems are being deleted.
- Caller info panel shows identity (title) / situation / signature
  (quoted detail) / secret_want (amber italic, host-only).
- Removed .energy-dot, .shape-badge, .info-badge.shape/energy/emotion
  CSS rules. Added .caller-identity and .caller-secret-want.
2026-04-05 04:26:04 -06:00
luke effd52a95c Populate CallRecord from new slim caller backgrounds
Replace the CallerBackground isinstance branching in _summarize_ai_call
with direct reads from the new slim caller background dict
(emotional_register, situation, specific_details). Decouples CallRecord
population from caller_styles/caller_shapes so those fields can be
deleted in a later commit.

CallRecord schema preserved for backward-compat with existing
session_checkpoint.json. topic_category, emotional_state, energy_level
are now always empty strings (obsolete under slim architecture).
2026-04-05 04:21:05 -06:00
luke 76feeb22a0 Collapse caller generation to single path, remove feature flag 2026-04-05 04:05:09 -06:00
luke 3f85ee9140 Add sample caller dialogue generation script + 10 validation samples 2026-04-05 03:40:07 -06:00
luke 604769a949 Tighten slim prompt: hard-ban stage directions 2026-04-05 03:15:53 -06:00
luke 969f8fb2fd Wire batch gen into Session.reset behind CALLER_REDESIGN flag 2026-04-05 03:06:42 -06:00
luke d4f1043be1 Add feature flag for slim caller path (haiku-4.5) 2026-04-05 03:02:45 -06:00
luke a18e1329f3 Add slim caller prompt builder (~400 tokens) 2026-04-05 03:01:16 -06:00
luke 2fd027b82a Add writer for promoted tier-2 regulars 2026-04-05 02:56:20 -06:00
luke 0c5d36d182 Strip markdown fences in _call_sonnet response 2026-04-05 02:46:38 -06:00
luke d7e475331d Add promotion gate for tier-2 regulars 2026-04-05 02:43:32 -06:00
luke e4a18e13ab Archive non-Silas regulars for redesign cutover 2026-04-05 02:40:28 -06:00
luke 470e92f8c4 Add Regular dataclass + lore file loader 2026-04-05 02:36:25 -06:00
luke b792c3cca0 Strip markdown fences in parse_batch_response 2026-04-05 02:31:17 -06:00
luke baa6db5aa2 Add batch generation function using sonnet-4.6 2026-04-05 02:25:24 -06:00
luke dec6211f7c Add batch prompt builder for caller_gen 2026-04-05 02:23:34 -06:00
luke ade5510bd5 Add voice roster validator for caller_gen 2026-04-05 02:22:10 -06:00
luke f56ca3d4bc Add CallerIdentity dataclass and batch JSON parser 2026-04-05 02:14:58 -06:00
lukeandClaude Opus 4.6 a196e8f088 Add caller generation redesign implementation plan
22 tasks across 5 phases: scaffolding, regulars v2, main.py integration
behind feature flag, user validation gate, then deletion of ~2000 lines.
TDD where possible; manual verification of LLM outputs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 02:09:27 -06:00
lukeandClaude Opus 4.6 ca7e82f19f Add missing InternService.new_show() method
Session.reset() calls intern_service.new_show() to reset per-show state,
but the method was never defined (added in 376265e without the impl).
Clears pending_interjection, pending_sources, and research_cache.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 02:02:53 -06:00
lukeandClaude Opus 4.6 1b988a0303 Ignore .worktrees/ directory
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 01:58:32 -06:00
lukeandClaude Opus 4.6 fa491c1c27 Add caller generation redesign plan
Two-stage architecture: sonnet-4.6 pre-gen batch for rich identities,
haiku-4.5 for live dialog. Replaces static pools + 9-model routing +
scoring/weighting scaffolding. Introduces tiered regulars system with
Silas as canonical (lore in Obsidian vault) and hard-gated arc regulars.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 01:57:32 -06:00
lukeandClaude Opus 4.6 486842d2d1 Add cost dashboard frontend: HTML, CSS, JavaScript
- costs.html: summary cards, 4 Chart.js charts, expensive calls table,
  sessions table with drill-down, session detail view
- costs.css: dark theme matching existing style.css variables, responsive
  grid layout, card/table/chart styling
- costs.js: parallel API fetching, Chart.js rendering (timeline, doughnut,
  bar, session bars), session detail drill-down, period tab filtering

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 23:58:22 -06:00
lukeandClaude Opus 4.6 9c9f5c6602 Add cost dashboard API endpoints and page route
7 new endpoints: summary, timeline, models, categories, sessions,
session detail, expensive calls. All support period filtering.
Page route at /costs serves costs.html before the catch-all.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 23:49:42 -06:00
lukeandClaude Opus 4.6 73d555b269 Dual-write cost tracker records to SQLite
Each LLM/TTS call now writes to both in-memory records and SQLite
via cost_db. Session totals updated on save(). All DB writes wrapped
in try/except to never break live shows.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 23:47:41 -06:00
lukeandClaude Opus 4.6 283e4a2f3c Fix cost_db review issues: thread safety, indexes, consistent filtering
- Add check_same_thread=False for background thread access
- Add standalone timestamp index for non-session queries
- Join to sessions table for consistent period filtering in get_models/categories/expensive_calls
- Wrap record imports in try/except with .get() for timestamp
- Add avg_cost_per_session to get_summary()

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 23:46:40 -06:00
lukeandClaude Opus 4.6 0d2a0e42a8 Add SQLite cost analytics database module
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 23:39:57 -06:00
lukeandClaude Opus 4.6 376265eec7 Show quality fixes + preflight check
Ep47 post-mortem: fixed theme ignored by callers (backgrounds now
regenerate when theme is set), style-to-model race condition (fallback
to sonnet instead of pool[0]), removed bad pronunciation fixes, added
age-awareness to voice matching, raised MIN_RESPONSE_WORDS to 50.

Swapped problematic model mappings: conspiracy→qwen, know_it_all→mistral,
quiet_nervous→llama, emotional→kimi.

Added GET /api/show/preflight endpoint with 4 checks: model diversity,
theme penetration, voice-age alignment, response coherence (2-exchange
simulation of all callers). Frontend preflight modal with expandable
check cards.

Fixed active caller button not highlighting (moved highlight code before
potentially-failing caller info panel code).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 01:17:34 -06:00
lukeandClaude Opus 4.6 f3c91fc385 Devon personality + Whisper name fix + music vocal filtering
- Devon: more conversational when addressed directly (500 tokens, 3-5 sentences)
- Devon: monitor prompt rewritten to encourage more contributions
- Devon: polling interval 15s (was 30s), removed 2-message minimum
- Whisper: no fuzzy name matching for 3-char names, require first letter match
- fetch_music.py: post-fetch vocal detection filter using musicinfo tags
- scan_music_vocals.py: new script to scan existing library for vocal tracks

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 23:59:03 -06:00
lukeandClaude Opus 4.6 c69c2ad532 Fix tonight's show issues: Whisper bias, boring callers, Devon, short responses
- Remove caller names from Whisper hint (was corrupting transcriptions)
- Background gen switched to Claude Sonnet 4.6 (cheap models = thin backgrounds)
- "WHAT MAKES A GOOD CALLER" rewritten with concrete examples
- Grok guardrails loosened (were cutting too much edge)
- Response length guidance added to caller prompt
- Retry under-20-word responses once for more detail
- Devon monitor softened from "default silence" to balanced
- Ban stalling phrases: "where was I", "as I was saying", etc.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 05:21:23 -06:00
lukeandClaude Opus 4.6 8dbbd92d3a Fix returning caller eligibility — 1+ calls, not 2+
The 2+ requirement created a catch-22: regulars couldn't return because they
needed 2 calls, but couldn't get a second call without returning. Dynamic
count already prevents flooding.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 03:42:53 -06:00
lukeandClaude Opus 4.6 fa36f8d184 Dynamic returning caller count — need 3+ eligible for variety
Only inject 2 returners if pool has 3+ eligible (so it's not the same every show).
With 2 eligible, inject 1. With 1 or 0, inject none.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 03:38:47 -06:00
lukeandClaude Opus 4.6 794ad98cf0 Replace music dropdown with genre quick-select buttons
- One-click genre buttons play random track from that genre
- Active genre highlighted, now-playing bar shows track name
- Only genres with tracks shown, crossfade on genre switch
- M key replays active genre or picks random

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 03:34:44 -06:00
lukeandClaude Opus 4.6 f5eabd7dc4 Add fetch_music.py (Jamendo API) + expand genre keywords
- Downloads instrumental tracks from Jamendo by genre (jazz, lofi, blues, ambient, etc.)
- Filters: no vocals, 60-300s, sorted by popularity
- Saves to music/ with genre tags, tracks attribution
- Add genre keywords: ambient, chill, acoustic, classical, country, electronic

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 03:18:26 -06:00
lukeandClaude Opus 4.6 f717edeacb Fix style map key mismatch — API uses 'map', frontend was using 'style_map'
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 02:45:21 -06:00
lukeandClaude Opus 4.6 56607879ee Fix style-matched dropdowns — populate from full model list, not just pool
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 02:42:53 -06:00
lukeandClaude Opus 4.6 fcefabdaee Expand style-matched routing to 10 models for maximum caller variety
- Grok 4.1 Fast: high_energy, bragger, comedian, small_town_gossip
- Grok 4 Full: confrontational (needs deep reasoning for arguments)
- Claude Sonnet 4.6: quiet_nervous, emotional (genuine vulnerability)
- Kimi K2: sweet_earnest (warm, creative, different texture than Claude)
- Mistral Large: deadpan, mysterious (dry, precise)
- DeepSeek Chat: angry_venting (raw, unfiltered rage)
- DeepSeek R1 Distill: oversharer, conspiracy (commits fully, no hedging)
- Qwen: storyteller, rambling (loves tangents and detail)
- Gemini 2.5 Pro: know_it_all (pedantic, cites sources)
- Llama 3.3 70B: world_weary, reluctant, first_time (casual, natural)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 02:40:09 -06:00
lukeandClaude Opus 4.6 58495d2c75 Fix stale model detection — validate against current OPENROUTER_MODELS
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 02:37:21 -06:00