◆ Fantasy Audiobook Studio
0story-seed1–5fantasy-novella2story-structure1, 4, 5fiction-workshop1, 4character-archetypes4prose-mechanics3, 4avoid-ai-writing6speech-flow7narration-prep8kokoro-narrate9listen-check10audiobook-assemble

Stage 9 · Voice · ours

listen-check

A robot proof-listener. It transcribes the finished audio and compares it with the script, so problems come back as a short list of timestamps instead of 90 minutes of listening.

  1. 0Seed
  2. 1Story bible
  3. 2Structure
  4. 3Draft
  5. 4Edit
  6. 5Reader test
  7. 6Speech flow
  8. 7Narration prep
  9. 8Narrate
  10. 9Listen check
  11. 10Assemble & publish

When it runs

After narration, before the audiobook is assembled, and after any re-render.

What goes in

The chapter audio and its script

What comes out

A list of moments to listen to (chapter, time, what was expected, what was heard)

Who does the work

  • Script transcribes the audio with Whisper, compares word by word, and checks pace and stress
  • Claude double-checks each flag with a second model and turns confirmed problems into training cases
  • Author listens to the short list and confirms

Step by step

  1. Transcribe each chapter with a speech-recognition model running on the laptop.
  2. Line the transcript up against the script, word by word.
  3. Flag skipped, swapped or garbled words, odd pacing, and flat short lines.
  4. Re-check each flag with a second, more careful model.
  5. Hand over the remaining timestamps; confirmed problems become training cases.

Example

Before

92 minutes of audio. Is anything wrong?

→
After

ch06 1:05: expected "A wall.", heard "A while." (both models agree)

About 90 raw flags in the whole book became 2 after filtering and a second opinion. One was a real problem nobody had reported.

In this project

Runs after narration, before assembling. Whisper (base.en, local, about 6× real time, so ~16 minutes per book) transcribes the audio, and the transcript is aligned word by word with the script. Skipped, swapped or garbled words come back as timestamps. Two cheap checks run alongside it: words per second per paragraph (rushed or garbled audio), and Kokoro's own phonemes for bare short lines (no main stress means a flat ending). Known Whisper noise is filtered out: digits for number words, words echoed at its 30-second window joins, dropped "the", and spelling variants. Each finding is double-checked with the slower small.en model before the author is asked to listen, and confirmed problems become training cases with the clip already cut.

From the books

  • First run, chapter 1: 8 raw flags. Three were numbers ("31" for "thirty-one"), and five were doubled words ("someone someone"). Re-transcribing short clips with both models showed each word only once, so the doubling was Whisper, not Kokoro. Both filters went into the script; chapter 1 now reports 0 issues.
  • Whole book, first run: about 90 raw flags. Sound-based filters (homophones, British spelling, names compared by phonemes) and the automatic second opinion from small.en brought it down to 2 word flags in 92 minutes, plus a few pace and stress notes. One was real: in chapter 6, "A wall." is heard as "A while." It became training case 06, the first problem the listener found that nobody had reported.
  • Limit: Whisper prefers familiar spellings, so it confirms that names aren't badly wrong, not that the stress is perfect. Names are still checked by ear once, in narration-prep.

Training cases

For the curious: the instructions Claude reads (SKILL.md)

Claude uses this skill when: Automatic listener for rendered audiobook chapters: transcribes the audio with Whisper (locally, CPU) and compares it with the narration script to find skipped, swapped or garbled words and misheard names, flags paragraphs spoken unusually fast or slow, and finds short lines with no main stress. Use after kokoro-narrate, before audiobook-assemble, when the user asks to check, QA or proof-listen the audio, or wants to find audio problems without listening to everything.

Listen Check

A human can't re-listen to 90 minutes after every change. This skill does a first listen automatically, so the author only has to hear a short list of moments with timestamps.

.venv/bin/python .claude/skills/listen-check/scripts/listen.py books/<slug>                    # all chapters, ~6x real time
.venv/bin/python .claude/skills/listen-check/scripts/listen.py books/<slug> --chapters ch03    # one chapter
.venv/bin/python .claude/skills/listen-check/scripts/listen.py books/<slug> --reuse         # re-analyse stored transcripts (after tuning)

Each chapter gets audio/chNN.listen.json, which the website shows, plus a printed report.

Check How What it catches
words Whisper (base.en) transcribes the chapter; the transcript is aligned word by word with chNN.txt Kokoro skipping, swapping or garbling words; names written wrongly in the transcript
pace words per second for every paragraph, from the render cache a paragraph > 30 % faster or slower than the chapter's median: rushed, dragged or garbled audio
stress Kokoro's own phonemes for bare short lines (no tag or beat) lines with no main stress, which end flat (training cases 01 and 05)

Filtered out as Whisper noise: - Numbers written as digits ("31" for "thirty-one", "50s" for "fifties"). - Echoes: a word repeated where Whisper's 30-second windows meet. - Small words: a dropped "the", "a" or "and". - Spelling: British vs American variants (grey/gray, centre/center). - Sound-alikes, compared by Kokoro's phonemes rather than by letters: won/one, route/root, pale/pail.

Names are compared by sound. Whisper spells a name it doesn't know the way it sounds ("Elin" → "Ellen", "Mira" → "mere"). A name passes when what Whisper heard is closer to the glossary pronunciation than to Kokoro's own wrong default. That includes possessives like "Mira's".

Whisper wrote Verdict Why
mere, mirror (Mira) pass sounds like MEER-uh
Ellen (Elin) pass sounds like EL-in
Myra (Mira) flag that's Kokoro's default, MY-ruh
Eline (Elin) flag that's the wrong ih-LIN

Second opinion, built in. Every word flag is re-checked automatically. The slower small.en model transcribes that paragraph's own clean audio from the render cache, and the flag is kept only if both models agree. Dismissed flags stay in the JSON under dismissed, for transparency.

Limits. Whisper prefers familiar words and spellings. If it writes "Mira" correctly, Kokoro said something close to "Mira", but not necessarily with the right stress. So a clean report means no gross errors, not perfect pronunciation. Names are still best checked by ear once, when the glossary is made (narration-prep).

Workflow

  1. Run it after kokoro-narrate, before audiobook-assemble.
  2. The report only contains flags both models agree on. Tell the author the remaining flags as timestamps to listen to, such as "ch04 3:12: expected 'sequestered', heard 'sequenced'". Don't claim a problem is real until it's confirmed by ear or by both models.
  3. Every confirmed problem becomes a training case (see training/README.md). Cut the clip automatically: listen.py books/<slug> --clip ch04 190 197 -o training/NN-slug.heard.wav. Then fix it in the right skill: a pronunciation goes to narration-prep's glossary or rules, a phrasing problem to speech-flow.
  4. Re-render the paragraph, re-run the check on that chapter, then assemble and publish.

How the skills improve (shared loop)

  • Author reports → training case → before/after audio → the author picks → a rule in the right skill.
  • listen-check reports → confirmed by a second model or by ear → a training case the same way.
  • Rule tests: every lesson that the speech-flow scanner can detect has a test in training/rule-tests.yaml. Run python3 .claude/skills/speech-flow/scripts/test_rules.py after changing any rule. All tests must pass before publishing; the website shows the results.
  • Rule proposals: when two or more cases share a pattern that no rule covers yet, Claude drafts a rule, a scanner check and a test, and adds it to training/proposals.md with status proposed. It becomes a rule only when the author approves it. Taste ("does this pause feel right?") always stays with the author.

Other files in the skill

View the skill on GitHub →