Video HTML definitions
HyperFrames is HeyGen's open-source video rendering framework, licensed under Apache 2.0, defining videos as you would build a webpage. Write an HTML document and mark each element's timing and layer with data attributes; the framework captures it frame by frame and encodes it as MP4. It is a pipeline that runs locally or can be deployed as a rendering backend, not a video editor or cloud SaaS.
The mechanism is deterministic. Each frame's time is calculated with frame = floor(time × fps); the framework uses Chrome's beginFrame API to seek the page to that time, capture it and send it to FFmpeg for encoding. The same input always produces the same output, independent of real-time playback speed, so results are reproducible and can be version controlled.
The design targets AI agents. A composition is plain HTML that any LLM can generate without learning a proprietary format; the CLI is non-interactive by default, takes all input through flags, returns plain text and stops on errors. For coding agents, generating a video and generating a webpage are the same kind of task.
HTML composition
headless Chrome
Frame-by-frame seek & capture
FFmpeg encode
MP4
Write HTML. Render video. Built for agents.
Agent and CLI installation paths
The environment requires only Node.js 22 or later and FFmpeg. FFmpeg handles final frame encoding and must be installed on the system first; macOS can use brew install ffmpeg. After checking these requirements, choose a path according to whether you use an AI coding agent.
Path A: AI coding agent skill installation
For agents that support skills, such as Claude Code and Cursor, a single command adds the HyperFrames skill. The agent can then read HTML, generate compositions and invoke rendering.
# Add the HyperFrames skill to your coding agent
npx skills add heygen-com/hyperframesPath B: Manual CLI use
Use the CLI directly without an agent. init creates a project, preview opens a real-time browser preview with live reload, and render outputs MP4.
npx hyperframes init my-video
cd my-video
npx hyperframes preview # Live browser preview · live reload
npx hyperframes render # Render to MP4Six packages and the rendering pipeline
HyperFrames is a monorepo divided into packages, each responsible for part of the pipeline. Everyday use requires only the hyperframes CLI; the other packages support programmatic integration, custom engines or distributed rendering. These core packages are listed in the official documentation.
CLI · 01
hyperframes
Command-line tool
The entry package provides init, preview, render, add and other commands. It is non-interactive by default and can be fully driven by an agent.
Core · 02
@hyperframes/core
Types · Parsing · runtime
Type definitions, HTML parsing, runtime and linter. The shared foundation for all higher-level packages.
Engine · 03
@hyperframes/engine
Capture engine
A page-to-video capture engine that seeks and captures frames in headless Chrome.
Producer · 04
@hyperframes/producer
Rendering pipeline
The complete rendering pipeline connects capture and FFmpeg encoding to produce the final MP4.
Studio · 05
@hyperframes/studio
Visual editor
A browser-based visual composition editor for adjusting timelines and layers.
Lambda · 06
@hyperframes/aws-lambda
Distributed rendering
Distributed rendering on AWS Lambda splits long videos into segments for parallel capture, then merges them.
Composition data attributes
The video timeline is marked directly on HTML elements without using JSON. The official documentation lists these core data attributes, which the runtime reads during frame-by-frame rendering.
| Data attribute | Purpose |
|---|---|
data-composition-id | Mark the composition root and define the scope of a video. |
data-width / data-height | Canvas dimensions, such as 1920 × 1080. |
data-start | Element entry time in seconds. |
data-duration | Element duration in seconds. |
data-track-index | Layer order, determining element stacking. |
data-volume | Volume control for audio or video elements with sound. |
Frame capture for seekable animations
Frame-by-frame rendering requires animations that can be positioned precisely at any time. HyperFrames uses the Frame Adapter pattern to abstract this into adapters; any seekable runtime can connect. The official documentation lists six supported animation sources.
| Adapter | Typical use |
|---|---|
GSAP | Timeline animation, easing and complex orchestration. |
CSS | Transitions and keyframes for style-driven animation. |
WAAPI | Web Animations API for native browser timeline control. |
Anime.js | Lightweight JavaScript animation library. |
Lottie | Vector animation exported from After Effects. |
Three.js | WebGL 3D scenes and camera animation. |
These implementation points come from the official documentation and cover common timeline and determinism issues.
Point 01 · Seconds and fps frame conversion
data-start and data-duration use seconds; frame = floor(time × fps) converts them to frames. Decide on fps before arranging the timeline to keep entry points from falling on non-integer frames.
Source · Official documentation · Rendering pipeline
Point 02 · data-track-index layer order
Element stacking uses data-track-index, not CSS z-index. Assign background video, title cards and audio to different tracks for stable, predictable composition order during rendering.
Source · Official documentation · Data attributes
Point 03 · data-volume volume control
Use data-volume to control background music and video with sound, rather than rely on media-file loudness. Set each track individually when mixing multiple audio tracks to keep one from overpowering narration.
Source · Official documentation · Data attributes
Point 04 · preview and render sequence
npx hyperframes preview opens a real-time browser preview with live reload, showing HTML edits immediately. Check the timeline before running render to avoid waiting for full frame-by-frame capture.
Source · Official documentation · CLI
Point 05 · add catalog snippets
npx hyperframes add <component> installs ready-made catalog snippets into the project, including transitions such as flash-through-white. Using existing snippets is faster than writing animations from scratch.
Source · Official documentation · CLI
Point 06 · Determinism and version control
The same input always produces the same output. The composition HTML can therefore go into Git, and CI can reproduce renders; video-change reviews compare text diffs rather than binary files.
Source · Official documentation · Core concepts
Six-second intro MP4 rendering workflow
This example is a six-second product intro with a background video, a title card and a music track. Use the CLI to create a project, inspect its generated HTML composition, edit data attributes, preview it and render it. The workflow needs no proprietary timeline format; all timing is marked in HTML.
<!-- Mark timing and layers on each element with data attributes -->
<div id="stage" data-composition-id="launch" data-start="0"
data-width="1920" data-height="1080">
<video class="clip" data-start="0" data-duration="6"
src="intro.mp4" muted playsinline></video>
<h1 id="title" data-start="1" data-duration="4">Launch day</h1>
<audio data-start="0" data-duration="6" src="music.wav"></audio>
</div>
$ npx hyperframes init launch-intro
ok: ✓ created launch-intro/
$ cd launch-intro
data-composition-id="launch" · 1920×1080
track 0 video intro.mp4 start 0 duration 6
track 1 h1 Launch day start 1 duration 4
track 2 audio music.wav start 0 duration 6
$ npx hyperframes preview
ok: ▶ preview server · http://localhost:5173
Change the title data-start from 1 to 0.5; the browser redraws immediately.
$ npx hyperframes add flash-through-white
ok: ✓ added component · flash-through-white
$ npx hyperframes render --output demo.mp4
ok: ✓ wrote demo.mp4 · 6.0s · 1920×1080 · 30fps
hl: Run the same composition.html again; the output is identical frame by frame.
The timeline is marked in HTML, and fps determines the output. The same composition produces the same result every time it is rendered.
HTML timing and reproducible output
From init to demo.mp4, there is no proprietary timeline file. Timing uses data-start and data-duration on HTML elements; layers use data-track-index. Preview and rendering use the same HTML, so they do not diverge.
Deterministic rendering allows composition.html to go directly into Git. Team reviews of video changes inspect text diffs, and CI can reproduce bit-identical output. For AI agents, generating and modifying videos is the same kind of operation as generating and modifying webpages.
Environment requirements and rendering limits
HyperFrames pipeline integration
The CLI is the operating entry point; use the relevant packages for programmatic video generation, custom capture engines or rendering at scale. These extensions follow the official documentation.
Extension options
1. Edit visually with Studio. @hyperframes/studio provides a browser-based composition editor for adjusting timelines and layers visually, then exporting back to HTML.
2. Connect the pipeline programmatically. For on-demand backend video generation, use the @hyperframes/producer rendering pipeline and @hyperframes/engine capture engine to make HyperFrames a rendering backend.
3. Render across AWS Lambda. @hyperframes/aws-lambda splits long videos, captures segments in parallel and merges them, shortening overall rendering time for high-volume or real-time scenarios.
4. Use catalog snippets. Apply ready-made snippets such as the flash-through-white transition with npx hyperframes add <component>, then adjust data attributes. This is faster than writing animations from scratch.
5. Use a coding agent. Install the skill in Claude Code or Cursor with npx skills add heygen-com/hyperframes. Compositions are plain HTML, so an agent can generate and modify them directly, then call render for output.
Further reading
① hyperframes.heygen.com/introduction: Official documentation with complete coverage of core concepts, data attributes and the CLI.
② github.com/heygen-com/hyperframes: Source code, package structure and license (Apache 2.0).
Write HTML. Render video. Built for agents.