Current section
Files
Jump to
Current section
Files
README.md
# phx_demo
Automated, reproducible demo videos for Phoenix / LiveView apps.
Write a demo as an ExUnit test, run `mix demo.record`, and get a narrated
MP4: intro card, auto-zoom on what's being clicked, captions drawn into the
picture (plus a soft subtitle track), chapters, and an outro. You also get a
GIF, SRT/VTT files, and visual drift warnings when the UI changes. The browser
is driven by [phoenix_test_playwright](https://hex.pm/packages/phoenix_test_playwright),
so demos use the Ecto sandbox and seed data just like your feature tests.
> Status: private, not yet on Hex. MIT licensed.
```elixir
# test/my_app_web/demos/posts_demo_test.exs
defmodule MyAppWeb.Demos.PostsDemo do
use PhxDemo.Case
use MyAppWeb, :verified_routes
demo "create a post", %{conn: conn} do
MyApp.Blog.create_post(%{title: "Seeded post"})
conn
|> scene("The posts list")
|> visit(~p"/posts")
|> say("Here are the posts we already have.")
|> scene("Creating a post")
|> glide_click_link("New Post")
|> type_into({:label, "Title"}, "Recorded by phx_demo")
|> glide_click_button("Save Post")
|> assert_has("#flash-info", text: "Post created successfully")
|> spotlight("#posts")
|> say("And there it is, saved and listed.")
end
end
```
```sh
mix demo.record # every demo under test/
mix demo.record --gif # also render GIFs
```
Output goes to `tmp/demos/<module>/<demo>/`: `<demo>.mp4`, `.srt`, `.vtt`,
optionally `.gif`, plus `raw.webm`, `timeline.json` and `screens/`.
## Install
Requirements: Elixir 1.18+, Node.js, ffmpeg (`brew install ffmpeg` /
`apt-get install ffmpeg`), python3 with venv (for the default Piper voice),
and Postgres if your app uses Ecto.
With [Igniter](https://hexdocs.pm/igniter) (recommended):
```sh
# add {:igniter, "~> 0.8", only: [:dev, :test]} to your deps first, then:
MIX_ENV=test mix igniter.install phx_demo@github:thenatespack/phoenix_demo --only test
```
Then follow the printed steps:
```sh
npm --prefix assets install --save-dev playwright
npx --prefix assets playwright install chromium
mix phx_demo.piper.setup
mix assets.build
mix demo.record
```
The installer:
- adds `phoenix_test_playwright`;
- adds `preferred_envs` for the mix tasks;
- sets up test config (endpoint `server: true`, `:sql_sandbox`, `:phoenix_test` otp_app);
- adds the Ecto sandbox plumbing for LiveView (sandbox plug, `:user_agent`
in `connect_info`, and an `on_mount` hook that does nothing unless
`:sql_sandbox` is on);
- edits `test_helper.exs`;
- adds a guard for Phoenix 1.8's runtime.exs port config, which otherwise
overrides the test port;
- creates an example demo.
It's safe to run twice. Any edit it can't make safely becomes a warning
describing the manual step.
<details>
<summary>Manual setup</summary>
1. `mix.exs`: add `{:phoenix_test_playwright, "~> 0.19", only: :test, runtime: false}`
and `{:phx_demo, github: "thenatespack/phoenix_demo", only: :test}`, plus
`def cli, do: [preferred_envs: ["demo.record": :test, "phx_demo.piper.setup": :test]]`.
2. `config/test.exs`: `config :my_app, MyAppWeb.Endpoint, server: true`,
`config :my_app, :sql_sandbox, true`, `config :phoenix_test, otp_app: :my_app`.
3. Endpoint: `if Application.compile_env(:my_app, :sql_sandbox), do: plug Phoenix.Ecto.SQL.Sandbox`,
and `connect_info: [:user_agent, session: @session_options]` on the LiveView socket.
4. An `on_mount` hook that calls `Phoenix.Ecto.SQL.Sandbox.allow/2`, registered in
`live_view/0` of your web module. See `spike/demo_app/lib/demo_app_web/live_acceptance.ex`.
5. `test/test_helper.exs`:
```elixir
{:ok, _} = PhoenixTest.Playwright.Supervisor.start_link()
Application.put_env(:phoenix_test, :base_url, MyAppWeb.Endpoint.url())
ExUnit.start(exclude: [:phx_demo])
```
6. Wrap the top-level `http: [port: ...]` config in `config/runtime.exs` with
`if config_env() != :test do … end`.
</details>
If something's missing, the first demo fails with an explanation. For example:
"Asset /assets/css/app.css returned HTTP 404 … Run `mix assets.build`", or
"Narration: Piper isn't installed. Run `mix phx_demo.piper.setup`".
## Steps
All steps take and return `conn`, so they mix with PhoenixTest's `visit`,
`click_link`, `fill_in`, `assert_has` and the rest.
| Step | What it does |
|------|--------------|
| `scene(title)` | Starts a chapter. The previous scene's end state is screenshotted for drift checks |
| `say(text, wait: ms \| false, gap: ms)` | Narration: spoken audio, a burned-in caption and a subtitle cue. Pauses until the line has been spoken |
| `glide(target, steps: 25)` | Moves the cursor smoothly to the target. The view auto-zooms there |
| `glide_click_link(text)` / `glide_click_button(text)` | Glides to the target, clicks, then waits for LiveView |
| `type_into(target, text, speed: :human \| :fast \| :slow \| ms)` | Glides to the field and types one key at a time |
| `spotlight(target)` / `clear_spotlight()` | Dims the page around an element |
| `pause(ms)`, `mark(label)` | Pacing, and a timeline marker |
| `await_liveview()` | Waits until LiveView is connected and nothing is loading. Glides do this automatically |
| `hide_cursor()` / `show_cursor()` | Hides or shows the overlay cursor |
| `upload(target, path \| [paths], picker: false)` | Glides to a file input (or its label), clicks, attaches fixture files, waits for LiveView uploads to finish. See [File uploads](#file-uploads) |
| `drop_files(target, path \| [paths])` | Drags files onto a `phx-drop-target`: file icons fly to the zone, then a real drop is dispatched. See [File uploads](#file-uploads) |
| `await_uploads()` | Waits for in-flight LiveView uploads. The steps above (and every glide) already do |
| `use_camera(path, loop: true)` | The page's camera (`getUserMedia` video) shows this video. Calling it again switches the live stream |
| `use_camera_image(path)` | A still image as the camera feed (QR or document scanning) |
| `stop_camera()` | Ends the fake camera: the app's tracks fire `ended`, new requests fail with `NotFoundError` |
| `upload_audio(target, path, drop: false, audible: false)` | `upload/4` (or `drop_files/4` with `drop: true`) for an audio file, plus an `audio_input` timeline event. See [Audio input](#audio-input) |
| `use_microphone(path, loop: false, audible: false)` | The page's microphone (`getUserMedia` audio) plays this audio file when the app starts listening |
| `stop_microphone()` | Ends the fake microphone: the app's tracks fire `ended`, new requests fail with `NotFoundError` |
Targets: a CSS or Playwright selector string, `{:link, text}`, `{:button, text}`,
`{:label, text}`, `{:text, text}`, or `{:role, role, name}`.
## File uploads
```elixir
conn
|> visit(~p"/uploads")
|> upload("#upload-form input[type=file]", "report.pdf") # via the file input
|> upload({:label, "Photos"}, ["a.png", "b.png"], picker: true) # with a fake file picker
|> drop_files("#drop-zone", "avatar.png") # drag and drop
|> assert_has("#uploaded", text: "avatar.png")
```
- **`upload/4`**: the cursor glides to the input (a visually hidden input is
fine: it goes to the label), a click ripple shows, and the files are attached
with Playwright's `set_input_files`. Playwright never opens the OS dialog, so
`picker: true` shows a fake file-picker sheet with the chosen files
highlighted for about a second (`picker_ms:` to change it). It's off by default.
- **`drop_files/4`**: file icons labeled with the file names fly in from
off-screen to the zone (`flight_ms:`, default 900) while the cursor glides
there, then `dragenter`, `dragover` and `drop` are dispatched on the target
with a `DataTransfer` built in the page from the files' real bytes. The target
is the element with `phx-drop-target`. Bytes travel base64-encoded to the
page, so use small fixtures.
- **Waiting**: both steps then wait until no file input has
`data-phx-preflighted-refs` different from `data-phx-done-refs` (LiveView's own
"still uploading" test) and no `phx-*-loading` class remains. There are no
sleeps, so the next step sees the finished entries. Every `glide` and
`glide_click_*` waits the same way. This covers `auto_upload: true` and
uploads started by submitting the form.
- **Fixtures** resolve from `test/demos/fixtures/` (`config :phx_demo,
:fixtures_dir, "path"`). `"report.pdf"` and `"fixtures/report.pdf"` are the
same file; absolute paths work too. `PhxDemo.Case` checks every path written
literally in a demo's `upload`/`drop_files` calls before the browser starts, so
a missing file fails the run with a message naming it. Paths computed at runtime
are checked when the step runs. The library ships no fixtures: bring your own
(the sample app has a tiny PDF, PNG and CSV in
`spike/demo_app/test/demos/fixtures/`).
- **Timeline**: each upload records an `upload` event in `timeline.json`:
`mode` (`"input"` or `"drop"`), `target`, `selector`, `box` (the target's
position and size), `files` (`[{name, size}]`) and `total_bytes`. Auto-zoom
treats it like a glide to `box`, so the view holds on the input or drop zone.
Captions and your own post-processing can read the file names from it.
- In a demo, `upload/3,4` replaces `PhoenixTest.upload/3,4,5` (same name,
different job). Call `PhoenixTest.upload/4` by its full name if you need that
one.
## Camera input
Apps that use the webcam (video calls, photo capture, QR scanning) get a
believable camera from a fixture, no hardware needed:
```elixir
use PhxDemo.Case, camera_fixtures: ["test/fixtures/presenter.mp4", "test/fixtures/qr.png"]
demo "scan a ticket", %{conn: conn} do
conn
|> use_camera_image("test/fixtures/qr.png")
|> visit(~p"/scan")
|> glide_click_button("Start scanning")
|> assert_has("#result", text: "TICKET-42")
|> use_camera("test/fixtures/presenter.mp4") # switch the live stream
|> stop_camera() # the app sees the track end
end
```
How it works: a JS stand-in overrides `navigator.mediaDevices.getUserMedia`
for video. The fixture is drawn on a canvas and `canvas.captureStream()` is the
camera, so the app's own `<video>` preview shows it and gets recorded.
Fixtures reach the page through a Playwright route on your app's origin
(`/__phx_demo_camera/...`), so they work with any CSP that allows `'self'` and
`blob:` media. Camera permission is granted on demo browser contexts. In a plain
`test`, tag `browser_context_opts: [permissions: ["camera"]]` if your app checks
`permissions.query`.
- Images and `.webm`/`.ogv` videos are used as they are. Playwright's Chromium
can't play H.264, so any other video (mp4, mov, ...) is converted with ffmpeg
to VP8 WebM at the viewport size and 30 fps (audio dropped) and cached in the
system temp dir.
- `camera_fixtures:` paths are checked in pre-flight, before the browser starts.
The steps also fail fast on a missing file.
- Each call records a `camera_input` timeline event (`action: "start"` with
`file`, `kind`, `loop`; or `action: "stop"`). Spotlight or glide to the
preview element and auto-zoom frames it.
- `use_camera(path, audio: true)` also makes the video's soundtrack the
microphone (see [Audio input](#audio-input)). Screen sharing
(`getDisplayMedia`) isn't simulated.
- A fixture set before `visit/2` persists across full page loads. Streams
requested before `use_camera/3` was called aren't affected.
## Audio input
Voice notes, speech-to-text and audio uploads need audio coming in. Two steps
supply it from a file:
```elixir
demo "send a voice note", %{conn: conn} do
conn
|> visit(~p"/uploads")
|> upload_audio("#voice-note", "hello.wav") # an audio file upload
|> use_microphone("question.wav", audible: true) # the app's mic hears this
|> glide_click_button("Record") # ...when it starts listening
|> stop_microphone()
end
```
- **`upload_audio(target, path, opts)`** is `upload/4` (or `drop_files/4` with
`drop: true`) plus an `audio_input` event. Everything about uploads applies:
LiveView `allow_upload` and plain inputs, the wait for upload progress, the
fixtures folder.
- **`use_microphone(path, loop: false)`** extends the camera's JS stand-in (see
[Camera input](#camera-input)): `getUserMedia({audio: ...})` returns a
`MediaStream` made from the file with WebAudio, and `{video, audio}` requests
get both fake tracks. Playback starts when the app *requests* the microphone,
not when the step runs, as with a person who starts talking after pressing
Record. A non-looping file plays once per request. Calling the step again
switches the file, also for an app that is already listening.
`stop_microphone()` ends the tracks (`ended` fires) and new requests fail with
`NotFoundError`. Microphone permission is granted on demo browser contexts.
- **`use_camera(path, audio: true)`** uses the video's soundtrack as the
microphone (`stop_camera/0` stops it too).
- **Formats**: anything ffmpeg decodes. WAV, MP3, Ogg, Opus and FLAC are served
as they are (Chromium decodes them); other formats such as m4a/aac, or a
video's soundtrack, are converted to WAV with ffmpeg and cached in the system
temp dir.
- **Pre-flight**: paths written literally in `upload_audio` and `use_microphone`
calls are checked before the browser starts, like `upload` fixtures.
- **Hearing it in the video**: `audible: true` mixes the audio into the MP4 at
the moment it was attached or started playing (the `audio_input` event's
`at_ms`), through the same mix as narration and [music](#background-music).
The music ducks under it like under `say`. Input audio and narration shouldn't
overlap; the render logs a warning if they do. Off by default, since the
audio is test input, not part of the story. A looping microphone plays until
the next `use_microphone`, `stop_microphone` or the end of the video.
- **Timeline**: `audio_input` events have `mode` (`"upload"`, `"microphone"`,
`"camera"`), `action`, `file`, `audible`. An upload has `action: "upload"`
and `duration_ms` (ffprobe); a microphone has `"use"` when the step runs,
`"play"` (with `duration_ms`, `loop`) when the app starts listening, and
`"stop"`.
- The audio context needs a user gesture to run, so start listening from a
click (as a record button does). A page that requests the microphone on load
may get silence.
## Presentation settings
Set per module or globally (`config :phx_demo, <key>, value`):
```elixir
use PhxDemo.Case,
intro: %{title: "Posts", subtitle: "A two-minute tour"}, # true (default), false, or a map
outro: false,
captions: :both, # :both (default), :burned, :soft, false
zoom: 1.8, # true (default, max 1.6x), false, or a max factor
trim: true, # cut dead time before the first step / after the last
music: "priv/demo_music.flac", # see Background music
viewport: %{width: 1920, height: 1080},
output_dir: "priv/demos"
```
Any other option (e.g. `async`, `ecto_sandbox_stop_owner_delay`) goes to
`PhoenixTest.Playwright.Case`.
## Narration
[Piper](https://github.com/OHF-Voice/piper1-gpl) is the default: free, local
and open source. `mix phx_demo.piper.setup` installs it into a private
virtualenv under your user cache and downloads the default voice. To use a
different voice or provider:
```elixir
# config/test.exs
config :phx_demo, :tts, {PhxDemo.TTS.Piper, voice: "en_US-amy-medium"}
config :phx_demo, :tts, {PhxDemo.TTS.OpenAI, voice: "nova"} # OPENAI_API_KEY
config :phx_demo, :tts, {PhxDemo.TTS.ElevenLabs, voice_id: "..."} # ELEVENLABS_API_KEY
config :phx_demo, :tts, PhxDemo.TTS.MacSay # macOS only
config :phx_demo, :tts, false # captions only
```
Clips are cached by provider, options and text, so re-recording an unchanged
demo makes no API calls.
## Background music
Bring your own track. Nothing is bundled, so there are no licensing questions.
```elixir
use PhxDemo.Case,
music: "priv/demo_music.flac", # path, or false (default) for none
music_volume: -18 # dB, default -18
# or globally: config :phx_demo, :music, "priv/demo_music.flac"
```
- Any format ffmpeg reads works (mp3, wav, flac, ogg, m4a/aac). WAV or FLAC is
recommended so the music is compressed only once, when the MP4 is encoded.
MIDI files work too, and so does a track generated by ElevenLabs; see below.
- Music plays whenever a track is configured. `music: false` turns it off,
including a global default, for one demo. There's no separate on/off switch.
- A short track loops; a long one is cut at the video's end. It fades in over
1s and out over 2s (less on very short videos).
- While a `say` line is spoken the music ducks by 10 dB, with 150 ms ramps.
The dips come from the `say` windows in the timeline, so they are
deterministic.
- Works with `tts: false` (no narration, so no ducking) and with narration.
- Relative paths resolve from the directory where `mix demo.record` runs. A
missing file fails the render with a clear message.
### MIDI music
`music:` also accepts a `.mid` or `.midi` file. It is rendered to WAV with
[fluidsynth](https://www.fluidsynth.org/) before the video is encoded, then
goes through the same loop/trim, fade and ducking pipeline as any other track.
```elixir
use PhxDemo.Case,
music: "priv/theme.mid",
soundfont: "priv/GeneralUser-GS.sf2" # optional, see below
# or globally: config :phx_demo, :soundfont, "priv/GeneralUser-GS.sf2"
```
- Install fluidsynth: `brew install fluid-synth` (macOS) or
`apt-get install fluidsynth fluid-soundfont-gm` (Debian/Ubuntu). When the
music is MIDI, the pre-flight check fails fast if fluidsynth isn't on PATH
or no soundfont is found. Non-MIDI music never needs it.
- Soundfont lookup order: the `soundfont:` setting, then the one downloaded by
`mix phx_demo.soundfont.setup`, then common system locations
(`/usr/share/sounds/sf2/FluidR3_GM.sf2`, `/opt/homebrew/share/soundfonts/`,
...). A `soundfont:` that points at a missing file is an error.
- No soundfont is bundled. `mix phx_demo.soundfont.setup` downloads
[GeneralUser GS](https://www.schristiancollins.com/generaluser) (about
32 MB, free for private and commercial use under its own license) from the
author's GitHub repository into your user cache, verified by SHA-256.
- The WAV is cached in `<output_dir>/.tts_cache`, keyed on the MIDI file's
contents, the soundfont (path, size, modified time) and the sample rate
(44.1 kHz), so re-recording doesn't run fluidsynth again.
- The MIDI's tempo isn't changed to fit the video. A short piece loops and a
long one is cut at the video's end, like any other track.
### Generated music (ElevenLabs)
Instead of a file, `music:` can be a prompt. The track is composed with the
[ElevenLabs Music API](https://elevenlabs.io/docs/api-reference/music/compose)
before the video is encoded, then goes through the same loop/trim, fade and
ducking pipeline as any other track.
```elixir
use PhxDemo.Case,
music: %{
elevenlabs: "calm lo-fi piano with soft brushed drums",
length_ms: 30_000, # 3_000..600_000, default 30_000 (short tracks loop)
instrumental: true, # default true: no vocals
model_id: "music_v1", # optional, default: the API's default
output_format: "mp3_44100_192" # optional MP3 format, default mp3_44100_128
}
```
#### Presets
Don't want to write a prompt? Use a preset name instead. The other options
(`length_ms`, `instrumental`, ...) still apply.
```elixir
use PhxDemo.Case, music: %{elevenlabs: :lofi}
use PhxDemo.Case, music: %{elevenlabs: :corporate, length_ms: 60_000}
```
| Preset | Sound |
|---------------|-------|
| `:lofi` | Chill lo-fi hip hop, dusty drums, Rhodes, 80 BPM |
| `:corporate` | Upbeat corporate pop, acoustic guitar, claps, 110 BPM |
| `:tech` | Minimal electronic, pulsing arpeggio, sub bass, 118 BPM |
| `:ambient` | Slow evolving pads, no percussion |
| `:piano` | Gentle solo piano, 75 BPM |
| `:acoustic` | Fingerpicked folk guitar, shaker, glockenspiel, 95 BPM |
| `:jazz` | Jazz trio, brushed drums, upright bass, 100 BPM |
| `:synthwave` | 1980s synthwave, gated snare, analog bass, 100 BPM |
| `:cinematic` | Hopeful strings and piano underscore |
| `:funk` | Light funk, slap bass, wah guitar, horn stabs, 105 BPM |
| `:playful` | Explainer-video marimba, ukulele, pizzicato, 115 BPM |
| `:chiptune` | Cheerful 8-bit, 120 BPM |
| `:deep_house` | Mellow deep house, 120 BPM |
| `:classical` | Light string quartet, 90 BPM |
| `:hip_hop` | Polished instrumental hip hop, 808s, 90 BPM |
| `:rock` | Upbeat indie rock, chiming guitars, 125 BPM |
Every preset adds the same instruction: steady background music for a demo
under a voice-over, with no vocals, drops or sudden builds.
`PhxDemo.MusicGen.presets/0` lists the exact prompts. An unknown preset fails
the pre-flight check and lists the valid names.
- Set `ELEVENLABS_API_KEY` (the same variable the ElevenLabs narration adapter
uses), or `config :phx_demo, :elevenlabs_api_key, "..."`. The key is never
written to `timeline.json`.
- The pre-flight check fails fast if the setting is invalid or no key is set.
- The MP3 is cached in `<output_dir>/.tts_cache`, keyed on the prompt and
options, so re-recording an unchanged demo makes no API call. Change the
prompt (or any option) to get a new track.
- Generation costs ElevenLabs credits, and their terms govern how you can use
the result.
## Visual drift
At the end of each scene a screenshot is compared with a baseline in
`test/demos/__baselines__/<os>/…`. Baselines are per OS because fonts render
differently on macOS and Linux. More than 0.1% of pixels changed (set with
`config :phx_demo, :drift_threshold, 0.5`) prints a warning with a diff image,
plus a `::warning` annotation on GitHub Actions. Drift **never fails** the run.
```sh
mix demo.record --update-baselines # accept the current UI
mix demo.record --no-drift
```
## Known limitations
- Video is recorded at the viewport's CSS-pixel size, so auto-zoom upscales
and zoomed text is slightly soft. Playwright's recorder doesn't capture at
device-pixel resolution.
- Drift compares whole screenshots, so timestamps and other dynamic content
can cause warnings. Keep them out of shot.
- One browser page per demo. Popups or extra tabs aren't supported yet.
## Repo layout
- `lib/`: the library
- `spike/demo_app`: a Phoenix 1.8 sample app used for end-to-end recording (and CI)
- `spike/FINDINGS.md`: the original feasibility spike