Local Text to Speech for Remotion: macOS say, ffprobe, and calculateMetadata (No API Key)

Published November 15, 2026 · 9 min read

Key takeaway

You can make narration for a Remotion video on your own Mac without an API key. Generate each clip with say, convert it to WAV with ffmpeg, read its length with ffprobe, and let calculateMetadata set each scene's length from that number. In our run, this kept all 14.29 s of speech in a 15.32 s video. A fixed 2 s per scene produced a 6.06 s video that cut most of the speech. Before you call say, check the voice name against say -v ?. An unknown voice does not cause an error. It quietly switches to a different voice and changes your timing.

Who this is for

This is for developers and solo creators who build videos with Remotion and want narration from a local TTS engine instead of a paid API. It covers macOS say only. We did not test other engines (see Limitations).

The problem: fixed scene lengths cut off speech

Remotion needs to know how many frames each scene lasts before it renders. If you guess, for example 2 s per scene, the guess will usually be wrong for spoken audio. Our baseline gave each of three scenes 2 s:

• The video was 6.06 s long, but the three clips held 14.29 s of speech.
• Each <Sequence> cuts off its audio at the end of the sequence, so the intro and wire speech stopped after 2 s.
• The last 0.4 s of the wire and outro clips fell past the end of the file, so the rendered video has no audio there.

The fix: measure each clip, then set frames from the measurement

The pipeline has four steps:

1. say writes AIFF.
2. ffmpeg converts it to 44.1 kHz mono WAV in public/audio.
3. ffprobe reads the duration, and the script writes it to data/manifest.json.
4. calculateMetadata turns those durations into frames and sets the composition length.

Script input (data/script.json)

{
  "voice": "Samantha",
  "scenes": [
    { "id": "intro", "text": "Local text to speech means the narration is generated on your own machine, with no API key." },
    { "id": "wire", "text": "Remotion needs to know how long each clip is, so we measure the audio and convert seconds to frames." },
    { "id": "outro", "text": "Then every scene lasts exactly as long as its voice track." }
  ]
}

TTS wrapper (scripts/tts.mjs)

// Local TTS via macOS `say`, converted to WAV with ffmpeg, duration measured with ffprobe.
import { execFileSync, spawnSync } from "node:child_process";
import fs from "node:fs";
import path from "node:path";
import { parseVoiceList } from "../src/timing.ts";

export const listVoices = () => parseVoiceList(execFileSync("say", ["-v", "?"], { encoding: "utf8" }));

export const probeDuration = (file) =>
  parseFloat(execFileSync("ffprobe", ["-v", "error", "-show_entries", "format=duration", "-of", "csv=p=0", file], { encoding: "utf8" }));

export function synth({ id, text, voice, outDir }) {
  if (!text || !text.trim()) throw new Error(`[${id}] empty text`);
  // `say` silently falls back to the default voice for unknown names (exit 0), so validate ourselves.
  if (!listVoices().includes(voice)) throw new Error(`[${id}] voice not installed: ${voice}`);
  fs.mkdirSync(outDir, { recursive: true });
  const aiff = path.join(outDir, `${id}.aiff`);
  const wav = path.join(outDir, `${id}.wav`);
  const t0 = performance.now();
  const r = spawnSync("say", ["-v", voice, "-o", aiff, "--", text]);
  if (r.status !== 0) throw new Error(`[${id}] say failed: ${r.stderr}`);
  execFileSync("ffmpeg", ["-y", "-v", "error", "-i", aiff, "-ar", "44100", "-ac", "1", wav]);
  fs.rmSync(aiff);
  const synthMs = performance.now() - t0;
  return { id, text, file: `audio/${id}.wav`, durationSec: probeDuration(wav), synthMs, bytes: fs.statSync(wav).size };
}

The voice check is necessary. In our run, say -v Nonexistent exited with code 0, wrote nothing to stderr, and used a different voice. That clip was 6.732 s long, while the same text with Samantha was 5.507 s.

Timing logic (src/timing.ts)

export type Clip = { id: string; durationSec: number; file: string; text: string };
export type Scene = Clip & { from: number; durationInFrames: number };

export const secondsToFrames = (sec: number, fps: number): number => {
  if (!Number.isFinite(sec) || sec <= 0) throw new Error(`invalid duration: ${sec}`);
  return Math.ceil(sec * fps);
};

// padSec = silence after each clip (so scenes do not butt together)
export const buildTimeline = (clips: Clip[], fps: number, padSec = 0.3) => {
  let from = 0;
  const scenes: Scene[] = clips.map((c) => {
    const durationInFrames = secondsToFrames(c.durationSec + padSec, fps);
    const s = { ...c, from, durationInFrames };
    from += durationInFrames;
    return s;
  });
  return { scenes, totalFrames: from };
};

// baseline: every scene gets a fixed length regardless of the audio
export const buildFixedTimeline = (clips: Clip[], fps: number, fixedSec: number) => {
  const d = Math.round(fixedSec * fps);
  const scenes: Scene[] = clips.map((c, i) => ({ ...c, from: i * d, durationInFrames: d }));
  return { scenes, totalFrames: d * clips.length };
};

export const parseVoiceList = (out: string): string[] =>
  out.split("\n").map((l) => l.match(/^(.+?)\s{2,}[a-z]{2}_[A-Z]{2}/)?.[1]?.trim()).filter((x): x is string => !!x);

Math.ceil rounds up, so a scene never ends a fraction of a frame before its speech does. The 0.3 s of padding keeps one scene's speech from running straight into the next. The unit tests check that durations of 0 s, negative values, and NaN are rejected.

Compositions (src/Root.tsx)

import React from "react";
import { Composition, CalculateMetadataFunction } from "remotion";
import { Video, VideoProps } from "./Video";
import { buildTimeline, buildFixedTimeline } from "./timing";
import manifest from "../data/manifest.json";

const FPS = 30;
const mode = (p: VideoProps) => p.mode;

const calc: CalculateMetadataFunction<VideoProps> = ({ props }) => {
  const clips = manifest.clips;
  const t = mode(props) === "fixed" ? buildFixedTimeline(clips, FPS, props.fixedSec) : buildTimeline(clips, FPS, 0.3);
  return { durationInFrames: t.totalFrames, props: { ...props, scenes: t.scenes } };
};

export const Root: React.FC = () => (
  <>
    {(["measured", "fixed"] as const).map((m) => (
      <Composition
        key={m}
        id={m}
        component={Video}
        width={640}
        height={360}
        fps={FPS}
        durationInFrames={30}
        defaultProps={{ mode: m, fixedSec: 2, scenes: [] } as VideoProps}
        calculateMetadata={calc}
      />
    ))}
  </>
);

Root.tsx imports data/manifest.json. Generate the manifest before you bundle or render. The value durationInFrames={30} is only a placeholder, and calculateMetadata replaces it.

Scenes (src/Video.tsx) and entry point (src/index.ts)

import React from "react";
import { AbsoluteFill, Audio, Sequence, staticFile } from "remotion";
import type { Scene } from "./timing";

export type VideoProps = { mode: "measured" | "fixed"; fixedSec: number; scenes: Scene[] };

export const Video: React.FC<VideoProps> = ({ scenes }) => (
  <AbsoluteFill style={{ background: "#111", color: "#fff", fontFamily: "sans-serif", fontSize: 28, padding: 40, justifyContent: "center" }}>
    {scenes.map((s) => (
      <Sequence key={s.id} from={s.from} durationInFrames={s.durationInFrames}>
        <AbsoluteFill style={{ padding: 40, justifyContent: "center" }}>{s.text}</AbsoluteFill>
        <Audio src={staticFile(s.file)} />
      </Sequence>
    ))}
  </AbsoluteFill>
);

import { registerRoot } from "remotion";
import { Root } from "./Root";
registerRoot(Root);

scripts/run-all.mjs ties the steps together. It runs the unit tests, synthesizes the clips, writes the manifest, runs the failure cases, renders both compositions with bundle, selectComposition, and renderMedia, and then checks the output with ffprobe and ffmpeg volumedetect. It is too long to include here, but it is part of the same project.

Results

All numbers come from the final run of node scripts/run-all.mjs. We did not listen to the output. The audio checks below use ffprobe and ffmpeg only.

**TTS (macOS say, Samantha, WAV 44.1 kHz mono).** *Audio sec* is the clip length reported by ffprobe. *Synth ms* is the wall-clock time for say plus the ffmpeg conversion. *WAV bytes* is the size of the output file.

The clips held 14.29 s of speech, and synthesis took about 2.0 s in total. That is about 7x faster than real time. Synthesis times changed from run to run. The intro took 685–830 ms across runs, and the overall ratio ranged from 6.8x to 7.2x.

Failure cases. *Case* is the input we tried. *Outcome* is what happened.

Renders (640x360, 30 fps, h264+aac). *Frames* is the number of video frames. *Video sec* is the file length. *Audio stream sec* is the length of the AAC stream. *Render sec* is the wall-clock time renderMedia took.

Mean volume of the last 0.4 s of each speech clip. *Speech sec* is the clip length. *Source dB* is the mean volume of that window in the WAV file. *Rendered dB* is the mean volume of the same window in the mp4, located with our own scene timeline. A value of -91 dB would mean silence.

In the measured render, the end of every clip has audio at about -20 dB. In the fixed render, the wire and outro clips end after the file does.

Test environment

• macOS 26.6.1 (Darwin 25.6.0), arm64, Apple M4
• Node v24.13.1
• Remotion 4.0.524 (@remotion/bundler and @remotion/renderer 4.0.524), React 19.2.0
• ffmpeg and ffprobe from Homebrew
• macOS say, voice Samantha (en_US)
• Docker was not used. Run on 2026-11-14.

Limitations

• We did not judge voice quality or how natural it sounds. Nobody listened to the audio.
• Only macOS say was tested. Piper, espeak-ng, Kokoro, and other engines were not tested, and we did not try a Linux or Docker setup.
• We did not check the license terms for publishing videos that use say voices. Check them before you publish.
• We used only 3 short English scenes. Longer scripts, other voices, and non-English text were not tested.
• The timing and speed numbers come from a few runs on one Apple M4 machine. They may differ on other hardware.
• The rendered tail volumes are 1.9–2.9 dB lower than the source WAV in the table above. We did not confirm the cause. AAC encoding, Remotion's audio mixing, or how the measurement windows line up are all possibilities. The check shows only whether speech is present, not its exact level.
• The volume check relies on our own scene timeline rather than detecting speech on its own. AAC encoder delay could shift the 0.4 s windows by a few milliseconds.
• We could not measure the end of the intro clip in the fixed baseline because that window overlaps the next scene's audio. The claim that the intro is cut off rests on the 6.06 s total length.

Reproduce

Prerequisites: a Mac with say and the Samantha voice installed (confirm with say -v ?), Node (we used v24.13.1), ffmpeg and ffprobe on your PATH, and the project dependencies installed: remotion, @remotion/bundler, and @remotion/renderer at 4.0.524, plus React 19.2.0 and TypeScript.

node --test test/timing.test.ts
node scripts/run-all.mjs
node_modules/.bin/tsc -p .

run-all.mjs writes the WAV files to public/audio, the durations to data/manifest.json, and the videos and results to artifacts/ (measured.mp4, fixed.mp4, results.json, and results.md). In our final run, 5 of 5 unit tests passed, tsc passed, and both renders completed.

Sources

This brief included no documentation URLs, so we have not linked any. The Remotion APIs used here are calculateMetadata, <Composition>, <Sequence>, <Audio>, and staticFile from remotion, bundle from @remotion/bundler, and selectComposition and renderMedia from @remotion/renderer. The official Remotion documentation describes each of them.

Last tested: 2026-11-14, Remotion 4.0.524