Publishing, website, reaper, and doc updates

Publish script, clip maker, website worker + data, reaper lua helpers,
audio settings, and CLAUDE.md doc reorg.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-09 23:39:57 -06:00
co-authored by Claude Opus 4.6
parent 44ac569fec
commit 61b3cba412
10 changed files with 398 additions and 127 deletions
+38 -34
View File
@@ -1,20 +1,5 @@
# AI Podcast - Project Instructions
## Git Remote (Gitea)
- **Repo**: `git@gitea-nas:luke/ai-podcast.git`
- **Web**: http://mmgnas:3000/luke/ai-podcast
- **SSH Host**: `gitea-nas` (configured in ~/.ssh/config)
- HostName: `mmgnas` (use `mmgnas-10g` if wired connection issues)
- Port: `2222`
- User: `git`
- IdentityFile: `~/.ssh/gitea_mmgnas`
## NAS Access
- **Hostname**: `mmgnas` (wireless) or `mmgnas-10g` (wired/10G)
- **SSH Port**: 8001
- **User**: luke
- **Docker path**: `/share/CACHEDEV1_DATA/.qpkg/container-station/bin/docker`
## Castopod (Podcast Publishing)
- **URL**: https://podcast.macneilmediagroup.com
- **Podcast handle**: `@LukeAtTheRoost`
@@ -42,8 +27,7 @@ Required in `.env`:
- ELEVENLABS_API_KEY (optional)
- INWORLD_API_KEY (for Inworld TTS)
## Post-Production Pipeline (added Feb 2026)
- **Branch**: `feature/real-callers` — all current work is here, pushed to gitea
## Post-Production Pipeline
- **Stem Recorder** (`backend/services/stem_recorder.py`): Records 5 WAV stems (host, caller, music, sfx, ads) during live shows. Uses lock-free deque architecture — audio callbacks just append to deques, a background writer thread drains to disk. `write()` for continuous streams (host mic, music, ads), `write_sporadic()` for burst sources (caller TTS, SFX) with time-aligned silence padding.
- **Audio hooks** in `backend/services/audio.py`: 7 tap points guarded by `if self.stem_recorder:`. Persistent mic stream (`start_stem_mic`/`stop_stem_mic`) runs during recording to capture host voice continuously, not just during push-to-talk.
- **API endpoints**: `POST /api/recording/start`, `POST /api/recording/stop` (auto-runs postprod in background thread), `POST /api/recording/process`
@@ -88,24 +72,44 @@ Required in `.env`:
- **Analytics**: Cloudflare Web Analytics (enable in Cloudflare dashboard, no code changes needed)
- **Deploy**: `npx wrangler pages deploy website/ --project-name=lukeattheroost --branch=main`
## Git Push
- If `mmgnas` times out, use the 10g hostname:
```bash
GIT_SSH_COMMAND="ssh -o HostName=mmgnas-10g -p 2222 -i ~/.ssh/gitea_mmgnas" git push origin main
```
## Hetzner VPS
- **IP**: `46.225.164.41`
- **SSH**: `ssh root@46.225.164.41` (uses default key `~/.ssh/id_rsa`)
- **Specs**: 2 CPU, 4GB RAM, 38GB disk (~33GB free)
- **Mail**: `docker-mailserver` at `/opt/mailserver/`
- **Manage accounts**: `docker exec mailserver setup email add/del/list`
- **Available for future services** — has headroom for lightweight containers. Not suitable for storage-heavy services (e.g. Castopod with daily episodes) without a disk upgrade or attached volume.
## Podcast Workflow
- Publishing pipeline: episodes go through Castopod, CDN, website, YouTube, and social
- Always check Python venv is active and packages are installed before running publish scripts
- Episode numbering must be verified against existing episodes
- Episode numbering: check Castopod for the latest episode number, don't hardcode
## Episodes Published
- Episode 6 published 2026-02-08 (podcast6.mp3, ~31 min)
## Scripts
- `publish_episode.py` — Transcribes audio, generates metadata (title, description, cover art), publishes to Castopod. Usage: `python publish_episode.py ~/Desktop/episode.mp3`
- `make_clips.py` — Two-pass clip extraction: fast Whisper transcription → LLM selects best moments → quality Whisper re-transcription for precise timestamps. Usage: `python make_clips.py ~/Desktop/episode.mp3 --count 3`
- `generate_milestone_images.py` — Generates social milestone images via Gemini Flash (requires GOOGLE_API_KEY)
- `post_milestone.py` — Posts milestone announcements to social platforms via Postiz
- `make_x_launch_assets.py` — Generates branded visual assets for X/Twitter (header, quote cards, intro/review graphics)
- `schedule_x_launch.py` — Schedules X/Twitter launch campaign posts via Postiz API
## Reaper Scripts
- `reaper/dialog_regions.lua` — Background script that polls `/tmp/reaper_state.txt` and creates colored regions (green=DIALOG, red=AD, blue=IDENT) as the backend writes state changes during recording
- `reaper/strip_silence_dialog.lua` — Post-production script: strips long silences from dialog regions, normalizes AD/IDENT/music volume, trims music to voice length with fade-out, mutes music during AD/IDENT regions
## Cost Dashboard
- **Route**: `/costs` — standalone analytics page, linked from control panel header
- **Database**: `data/costs.db` (SQLite) — aggregates all session cost data for cross-session queries
- **Data layer**: `backend/services/cost_db.py` — schema, JSON import, all query functions
- **Dual-write**: `cost_tracker.py` writes to both JSON (`data/cost_reports/`) and SQLite on every LLM/TTS call
- **API**: 8 endpoints under `/api/costs/` — summary, timeline, models, categories, sessions, session detail, expensive calls, TTS providers
- **Frontend**: `frontend/costs.html`, `frontend/css/costs.css`, `frontend/js/costs.js` — Chart.js for visualizations
- **Pricing**: Hardcoded in `cost_tracker.py` (`OPENROUTER_PRICING`, `TTS_PRICING`) — update when provider prices change
- **Not tracked yet**: SignalWire call costs
## Data Directory
State files (not config — these are written at runtime):
- `regulars.json` — Returning caller profiles (backgrounds, key moments, arc status, relationships)
- `used_topics_history.json` — Previously used caller topics to avoid repeats
- `session_checkpoint.json` — Current show session state (call history, caller queue)
- `publish_state.json` — Publishing pipeline progress per episode
- `intern.json` — Devon's lookup history
- `emails.json` — Listener email submissions
- `voicemails.json` — Listener voicemail submissions
## Personal
- Don't build anything until you have 95% clarity on what I want you to do. Ask clarifying questions until you reach 95% understanding of what I'm asking
- When working as a team, propose the plan before executing — don't just start building
- Flag trade-offs that affect show quality or listener experience rather than silently resolving them
+3 -2
View File
@@ -1,10 +1,11 @@
{
"input_device": 13,
"input_device": 14,
"input_device_name": "Babyface Pro (70793771)",
"input_channel": 1,
"output_device": 12,
"output_device": 13,
"output_device_name": "Radio Voice Mic",
"caller_channel": 3,
"devon_channel": 17,
"live_caller_channel": 9,
"music_channel": 5,
"sfx_channel": 7,
+2 -2
View File
@@ -1249,10 +1249,10 @@ def generate_clip_video_remotion(
]
try:
result = subprocess.run(cmd, capture_output=True, text=True, cwd=str(REMOTION_DIR), timeout=180)
result = subprocess.run(cmd, capture_output=True, text=True, cwd=str(REMOTION_DIR), timeout=600)
except subprocess.TimeoutExpired:
props_path.unlink(missing_ok=True)
print(f" Remotion render timed out (180s)")
print(f" Remotion render timed out (600s)")
return False
props_path.unlink(missing_ok=True)
+67 -2
View File
@@ -204,12 +204,12 @@ TRANSCRIPT:
"Content-Type": "application/json"
},
json={
"model": "anthropic/claude-3.5-sonnet",
"model": "anthropic/claude-sonnet-4.6",
"messages": [{"role": "user", "content": full_prompt}],
"max_tokens": 8192,
"temperature": 0
},
timeout=120
timeout=300
)
except requests.exceptions.Timeout:
print(f" Warning: Speaker labeling timed out for chunk {i+1}, using raw text")
@@ -1311,6 +1311,10 @@ def upload_to_youtube(audio_path: str, metadata: dict, chapters: list,
video_path = Path(audio_path).with_suffix(".yt.mp4")
# Convert MP3 + cover art to MP4 (pad to 1920x1080 for YouTube compatibility)
# Skip if video already exists and is non-trivial size (>1MB)
if video_path.exists() and video_path.stat().st_size > 1_000_000:
print(f" Using existing video: {video_path} ({video_path.stat().st_size / 1_000_000:.0f}MB)")
else:
print(" Converting audio to video...")
result = subprocess.run([
"ffmpeg", "-y", "-loop", "1",
@@ -1446,6 +1450,7 @@ def main():
parser.add_argument("--title", "-t", help="Override generated title")
parser.add_argument("--description", help="Override generated description")
parser.add_argument("--session-data", "-s", help="Path to session export JSON (from /api/session/export)")
parser.add_argument("--resume", action="store_true", help="Resume a failed publish — skip transcription/Castopod, continue from CDN/YouTube/social")
args = parser.parse_args()
audio_path = Path(args.audio_file).expanduser().resolve()
@@ -1493,6 +1498,66 @@ def main():
episode_number = get_next_episode_number()
print(f"Episode number: {episode_number}")
# --- Resume path: skip transcription + Castopod, pick up from CDN/YouTube/social ---
if args.resume:
castopod_step = _get_step_details(episode_number, "castopod")
if not castopod_step:
print(f"Error: No Castopod data in publish_state.json for episode {episode_number}. Nothing to resume.")
_cleanup_mysql_auth()
lock_fp.close()
LOCK_FILE.unlink(missing_ok=True)
sys.exit(1)
episode = {"id": castopod_step["episode_id"], "slug": castopod_step["slug"]}
print(f"Resuming episode {episode_number}: id={episode['id']}, slug={episode['slug']}")
# Load existing metadata from files saved during first run
transcript_path = audio_path.with_suffix(".transcript.txt")
chapters_path = audio_path.with_suffix(".chapters.json")
srt_path = audio_path.with_suffix(".srt")
# Build metadata from existing files; re-transcribe + gen only if missing
if chapters_path.exists():
print(" Loading existing chapters and metadata files")
with open(chapters_path) as f:
chapters_data = json.load(f)
# Reconstruct metadata from slug and chapters
slug = episode["slug"]
# Convert slug to title: "episode-49-foo-bar" -> "Episode 49: Foo Bar"
title_part = slug.replace(f"episode-{episode_number}-", "").replace("-", " ").title()
metadata = {
"title": f"Episode {episode_number}: {title_part}",
"description": f"Episode {episode_number} of Luke at the Roost.",
"chapters": chapters_data if isinstance(chapters_data, list) else chapters_data.get("chapters", []),
"thumbnail_text": title_part.upper()[:30],
}
else:
print("[1/5] Re-transcribing audio for metadata...")
transcript = transcribe_audio(str(audio_path))
metadata = generate_metadata(transcript, episode_number)
save_chapters(metadata, str(chapters_path))
if not transcript_path.exists():
labeled_text = label_transcript_speakers(transcript["full_text"])
with open(transcript_path, "w") as f:
f.write(labeled_text)
if not srt_path.exists():
generate_srt(transcript["segments"], str(srt_path))
if args.title:
metadata["title"] = args.title
if args.description:
metadata["description"] = args.description
srt_path = audio_path.with_suffix(".srt")
direct_upload = os.path.getsize(str(audio_path)) > CLOUDFLARE_UPLOAD_LIMIT
chapters_uploaded = True
transcript_uploaded = True
yt_video_id = None
# Jump to CDN upload (step 3.7)
# (fall through to the common CDN/publish/YouTube/social code below)
else:
# --- Normal (non-resume) path ---
# Guard against duplicate publish
if not args.dry_run:
exists = _check_episode_exists_in_db(episode_number)
+2 -1
View File
@@ -24,7 +24,8 @@ local last_state = ""
local transport_active = false
local function log(msg)
reaper.ShowConsoleMsg("[Regions] " .. msg .. "\n")
-- Silent by default — uncomment for debugging:
-- reaper.ShowConsoleMsg("[Regions] " .. msg .. "\n")
end
local function is_playing_or_recording()
+2 -1
View File
@@ -30,7 +30,8 @@ local BLOCK_SAMPLES = math.floor(SAMPLE_RATE * BLOCK_SEC)
local THRESHOLD = 10 ^ (SILENCE_DB / 20)
local MIN_VOICE_BLOCKS = math.ceil(MIN_VOICE_SEC / BLOCK_SEC)
local function log(msg)
reaper.ShowConsoleMsg("[PostProd] " .. msg .. "\n")
-- Silent by default — uncomment for debugging:
-- reaper.ShowConsoleMsg("[PostProd] " .. msg .. "\n")
end
---------------------------------------------------------------------------
+20 -3
View File
@@ -58,9 +58,26 @@ export default {
return new Response("Feed unavailable", { status: 502 });
}
// Plausible analytics proxy (bypass ad blockers)
// Umami analytics proxy (bypass ad blockers)
if (url.pathname === "/api/send" && request.method === "POST") {
const body = await request.text();
const resp = await fetch("https://plausible.macneilmediagroup.com/api/send", {
method: "POST",
headers: {
"Content-Type": "application/json",
"User-Agent": request.headers.get("User-Agent") || "",
"X-Forwarded-For": request.headers.get("CF-Connecting-IP") || request.headers.get("X-Forwarded-For") || "",
},
body,
});
return new Response(resp.body, {
status: resp.status,
headers: { "Content-Type": resp.headers.get("Content-Type") || "text/plain" },
});
}
if (url.pathname === "/p/script") {
const resp = await fetch("https://plausible.macneilmediagroup.com/js/script.file-downloads.hash.outbound-links.pageview-props.revenue.tagged-events.js");
const resp = await fetch("https://plausible.macneilmediagroup.com/script.js");
return new Response(await resp.text(), {
headers: {
"Content-Type": "application/javascript",
@@ -71,7 +88,7 @@ export default {
if (url.pathname === "/p/event" && request.method === "POST") {
const body = await request.text();
const resp = await fetch("https://plausible.macneilmediagroup.com/api/event", {
const resp = await fetch("https://plausible.macneilmediagroup.com/api/send", {
method: "POST",
headers: {
"Content-Type": "application/json",
+118 -1
View File
@@ -1,4 +1,121 @@
[
{
"title": "Thinking in English First",
"description": "When she heard about her father's diagnosis, she processed it in English first\u2014before responding to her mom in Czech. It felt like a betrayal of who she used to be.",
"episode_number": 49,
"clip_file": "clip-5-thinking-in-english-first.mp4",
"youtube_id": "LGqpEAxe754",
"featured": false,
"thumbnail": "images/clips/clip-5-thinking-in-english-first.jpg"
},
{
"title": "The April Fool's Divorce Text Disaster",
"description": "She thought sending her husband a divorce text as an April Fool's joke would be funny. Then his mom got involved and the casseroles started arriving. For 11 days straight.",
"episode_number": 48,
"clip_file": "clip-2-the-april-fool-s-divorce-text-disaster.mp4",
"youtube_id": "L9XAQgZeQo0",
"featured": false,
"thumbnail": "images/clips/clip-2-the-april-fool-s-divorce-text-disaster.jpg"
},
{
"title": "Caller's Daughter on Adult Website Revealed",
"description": "A caller discovers his daughter's secret online activities in the worst possible way. The details just keep getting more uncomfortable.",
"episode_number": 47,
"clip_file": "clip-3-caller-s-daughter-on-adult-website-revealed.mp4",
"youtube_id": "LiH6gVDTHcg",
"featured": false,
"thumbnail": "images/clips/clip-3-caller-s-daughter-on-adult-website-revealed.jpg"
},
{
"title": "The Taint Explanation",
"description": "Luke's caller had to explain what a taint is on live radio and it went exactly as uncomfortably as you'd imagine. This is why we can't have nice things.",
"episode_number": 46,
"clip_file": "clip-2-the-taint-explanation.mp4",
"youtube_id": "7qAhmnwUE1c",
"featured": false,
"thumbnail": "images/clips/clip-2-the-taint-explanation.jpg"
},
{
"title": "Bear Traps and Pit of Snakes",
"description": "Luke's home security system is straight out of Home Alone but way more dangerous. Swinging logs? Check. Pit of snakes covered with dust? BOOM! \ud83d\udc0d",
"episode_number": 44,
"clip_file": "clip-2-bear-traps-and-pit-of-snakes.mp4",
"youtube_id": "XBSEu2bFUAI",
"featured": false,
"thumbnail": "images/clips/clip-2-bear-traps-and-pit-of-snakes.jpg"
},
{
"title": "Roommate's Age Play Kink Takes Over Kitchen",
"description": "Roommate's age play kink is now front and center in the shared kitchen. Baby bottles by the coffee maker? Pacifiers on the counter? This living situation just got complicated.",
"episode_number": 43,
"clip_file": "clip-3-roommate-s-age-play-kink-takes-over-kitchen.mp4",
"youtube_id": "5cF_O2Gm9yM",
"featured": false,
"thumbnail": "images/clips/clip-3-roommate-s-age-play-kink-takes-over-kitchen.jpg"
},
{
"title": "Day Trading Away the Marriage",
"description": "She's been pretending to commute for 12 hours a day while secretly day-trading their life savings away in motels. Two years of lies and $145K gone.",
"episode_number": 42,
"clip_file": "clip-1-day-trading-away-the-marriage.mp4",
"youtube_id": "JeZ9OLGfW8A",
"featured": false,
"thumbnail": "images/clips/clip-1-day-trading-away-the-marriage.jpg"
},
{
"title": "The Snot Boogie Beer Money Thief",
"description": "He showed up to the barbecue claiming he's 'not like other guys' then literally ran off with everyone's beer money. The audacity is unmatched.",
"episode_number": 42,
"clip_file": "clip-2-the-snot-boogie-beer-money-thief.mp4",
"youtube_id": "taBAo8_YMkA",
"featured": false,
"thumbnail": "images/clips/clip-2-the-snot-boogie-beer-money-thief.jpg"
},
{
"title": "Aliens Chasing in Ford Trucks",
"description": "Glowing Ford trucks with shadowy drivers that aren't quite human. This caller's alien encounter story is straight out of a fever dream.",
"episode_number": 41,
"clip_file": "clip-3-aliens-chasing-in-ford-trucks.mp4",
"youtube_id": "Io8CzDtKGfA",
"featured": false,
"thumbnail": "images/clips/clip-3-aliens-chasing-in-ford-trucks.jpg"
},
{
"title": "Doctor Prescribes Prostate Stimulation",
"description": "Dale's doctor hands him a prostate stimulation pamphlet after his cancer diagnosis. Luke's medical advice? 'Stick some stuff up there and see what works.'",
"episode_number": 40,
"clip_file": "clip-3-doctor-prescribes-prostate-stimulation.mp4",
"youtube_id": "nQtwJpocFpY",
"featured": false,
"thumbnail": "images/clips/clip-3-doctor-prescribes-prostate-stimulation.jpg"
},
{
"title": "Open Marriage Backfired Spectacularly",
"description": "He agreed to an open marriage and now his wife is living her best life while he sits home alone every night. This call is PAINFUL to listen to.",
"episode_number": 39,
"clip_file": "clip-1-open-marriage-backfired-spectacularly.mp4",
"youtube_id": "-K-t7iijfGs",
"featured": false,
"thumbnail": "images/clips/clip-1-open-marriage-backfired-spectacularly.jpg"
},
{
"title": "Cat Burglar Bringing Home Stolen Goods",
"description": "This caller's cat has been stealing from the neighbors and bringing home cash and car keys. Where is this criminal mastermind getting this stuff?",
"episode_number": 39,
"clip_file": "clip-2-cat-burglar-bringing-home-stolen-goods.mp4",
"youtube_id": "JvgJWxFCBZk",
"featured": false,
"thumbnail": "images/clips/clip-2-cat-burglar-bringing-home-stolen-goods.jpg"
},
{
"title": "Second Baby Shower Entitlement Rant",
"description": "A second baby shower with a full registry? This caller is NOT having it and goes OFF about the entitlement of expecting gifts for baby number two.",
"episode_number": 39,
"clip_file": "clip-3-second-baby-shower-entitlement-rant.mp4",
"youtube_id": "NKt8NjDHKcg",
"featured": false,
"thumbnail": "images/clips/clip-3-second-baby-shower-entitlement-rant.jpg"
},
{
"title": "Cult Leader Realizes He's Been Manipulating People",
"description": "Cult leader calls in having a full existential crisis about his 'shared intimacy nights' and the manipulation tactics he's been using on his followers.",
@@ -55,7 +172,7 @@
},
{
"title": "Started a Fight and Can't Stop Reading About Wars",
"description": "A caller starts a fight with their partner and spirals into an obsessive deep-dive on historical wars. Luke tries to untangle the connection.",
"description": "",
"episode_number": 31,
"clip_file": "clip-3-started-a-fight-and-can-t-stop-reading-about-wars.mp4",
"youtube_id": "D2iWnSGQeow",
+1 -2
View File
@@ -92,8 +92,7 @@
}
}]
</script>
<script defer data-domain="lukeattheroost.com" data-api="/p/event" src="/p/script"></script>
<script>window.plausible = window.plausible || function() { (window.plausible.q = window.plausible.q || []).push(arguments) }</script>
<script defer data-website-id="ae816554-19c8-4e97-a2cb-d0226c27e9bc" data-host-url="/" src="/p/script"></script>
</head>
<body>
+66
View File
@@ -270,4 +270,70 @@
<changefreq>never</changefreq>
<priority>0.7</priority>
</url>
<url>
<loc>https://lukeattheroost.com/episode.html?slug=episode-39-st-patrick-s-day-chaos-and-caller-confessions</loc>
<lastmod>2026-03-18</lastmod>
<changefreq>never</changefreq>
<priority>0.7</priority>
</url>
<url>
<loc>https://lukeattheroost.com/episode.html?slug=episode-40-prostate-cancer-christmas-lights-and-potato-salad-betrayals</loc>
<lastmod>2026-03-19</lastmod>
<changefreq>never</changefreq>
<priority>0.7</priority>
</url>
<url>
<loc>https://lukeattheroost.com/episode.html?slug=episode-41-benny-s-creepy-vhs-tape-from-nowhere</loc>
<lastmod>2026-03-20</lastmod>
<changefreq>never</changefreq>
<priority>0.7</priority>
</url>
<url>
<loc>https://lukeattheroost.com/episode.html?slug=episode-42-peggy-s-day-trading-disaster-and-the-nonprofit-yacht</loc>
<lastmod>2026-03-21</lastmod>
<changefreq>never</changefreq>
<priority>0.7</priority>
</url>
<url>
<loc>https://lukeattheroost.com/episode.html?slug=episode-43-cousin-rufus-and-the-locksmith-s-wild-weekend</loc>
<lastmod>2026-03-23</lastmod>
<changefreq>never</changefreq>
<priority>0.7</priority>
</url>
<url>
<loc>https://lukeattheroost.com/episode.html?slug=episode-44-floyd-s-fence-and-the-cattle-conspiracy</loc>
<lastmod>2026-03-24</lastmod>
<changefreq>never</changefreq>
<priority>0.7</priority>
</url>
<url>
<loc>https://lukeattheroost.com/episode.html?slug=episode-45-potato-salad-aliens-and-the-dripping-faucet-of-doom</loc>
<lastmod>2026-03-26</lastmod>
<changefreq>never</changefreq>
<priority>0.7</priority>
</url>
<url>
<loc>https://lukeattheroost.com/episode.html?slug=episode-46-butchers-wreak-forever-and-other-late-night-confessions</loc>
<lastmod>2026-03-30</lastmod>
<changefreq>never</changefreq>
<priority>0.7</priority>
</url>
<url>
<loc>https://lukeattheroost.com/episode.html?slug=episode-47-clarence-randy-and-the-onlyfans-daughter</loc>
<lastmod>2026-03-30</lastmod>
<changefreq>never</changefreq>
<priority>0.7</priority>
</url>
<url>
<loc>https://lukeattheroost.com/episode.html?slug=episode-48-potato-salad-legacy-and-the-april-fool-s-apocalypse</loc>
<lastmod>2026-04-01</lastmod>
<changefreq>never</changefreq>
<priority>0.7</priority>
</url>
<url>
<loc>https://lukeattheroost.com/episode.html?slug=episode-49-silas-s-shared-intimacy-night-and-four-brave-souls</loc>
<lastmod>2026-04-05</lastmod>
<changefreq>never</changefreq>
<priority>0.7</priority>
</url>
</urlset>