BLOG

Tech Breakdown 002 | hyperframes: Write HTML, Get Video — HeyGen Open-Sources Video Rendering

Kael Zhang
AIOpen SourceVideo
广告 · Advertisement

1. What Is This

hyperframes is an open-source video generation framework from HeyGen. In one sentence: Write HTML/CSS, render it as an MP4 video.

It turns “making a video” into “building a webpage”. If you know frontend development, you can make videos — title animations, data charts, brand motion effects, all written in HTML + CSS, and hyperframes handles rendering them into a deterministic MP4.

Key facts:

  • GitHub repository: heygen-com/hyperframes, ~46k stars, created in March 2026, reached this scale in half a year, licensed under Apache 2.0
  • Implemented in TypeScript, requires Node.js ≥ 22, npm package name is simply hyperframes
  • Tagline: “Write HTML. Render video. Built for agents.” — the last part is the key: it’s designed specifically for AI coding agents
  • Three usage modes with the same engine: local CLI for personal use, adding skills for AI agents, or serving as the rendering core for hosted platforms

2. Core Mechanism: Why “Writing HTML” Equals “Making Video”

The key to this design lies in three technical decisions.

First, video = snapshots of a webpage across a timeline. Traditional video is a sequence of frames; hyperframes treats every frame as “the state of an HTML page at a point in time”. It captures each frame of your CSS transition animations, encodes them, and outputs MP4. The animation effect you see in the browser is exactly what you get in the final video — what you see is what you get, no mental translation of timelines required.

Second, seekable animations. This is the core of the entire framework. Regular web animations just play through and can’t be reversed; hyperframes requires all animations to support “jumping to any point in time to get the current state”. Only with deterministic seekability does frame-by-frame rendering work — the same point in time always renders the same frame, eliminating the flickering and random element jumps that plague traditional screen-recording approaches.

Third, the Skills system is agent-friendly. It comes with 20 built-in skills that teach AI agents the full production workflow: “plan video → write valid HTML → implement seekable animations → add media → lint → preview → render”. The entry point is a routing skill /hyperframes — agents read it and know which workflow to call. Installation takes one line: npx skills add heygen-com/hyperframes, supported by Claude Code, Cursor, Gemini CLI, and Codex.

The direct result of this combination: “agents that can write webpages with AI” instantly gain video production capacity. No need to learn After Effects, no need to understand video encoding — however good your HTML is, that’s how good your video will be.

3. Technical Assessment: Highlights and Boundaries

Let’s start with the highlights — all solid engineering work:

  1. Deterministic rendering. The same code always produces the same video. This is a hard requirement for content production — if you change a font size and re-render, nothing else will mysteriously shift. Traditional “screen recording to video” solutions can’t do this.
  2. Mature skill design. It doesn’t just throw an API at agents — it teaches agents the “production process” itself (plan first, then write, then lint, then preview, then render). The README goes into extreme detail on “which skills to install, how to install them on demand, and how to handle non-interactive runs” — this is a project that’s been actually used and has real battle scars.
  3. Smart ecosystem positioning. HeyGen is a commercial company making AI digital human videos. By open-sourcing the rendering engine and leaving the creation entry point to agents, it’s turning “every agent on the internet that can write HTML” into potential users of its ecosystem.

Now let’s be clear about the boundaries:

  1. It makes “graphic motion videos”, not “live-action footage”. Title animations, data visualizations, product intros, infographic motion — it does these fast and well; don’t expect it to generate real human live-action footage, that’s a completely different technical path.
  2. Rendering runs locally. Video rendering is CPU/GPU intensive. Long videos and high-resolution output will take time to render on your local machine — it’s not instant.
  3. Quality ceiling depends on your HTML skills. The framework is responsible for “stably turning webpages into videos”, but the design sense and animation timing of the page itself are still determined by the human (or agent) building it.
  4. Only six months old. The API is still iterating rapidly — be prepared to keep up with version upgrades if using it in production.

4. Value Judgment: Who Should Pay Attention, Who Can Wait

The real problem it solves: the production cost of video content. A product intro animation, a short data visualization clip — previously you’d need to open AE and wait for a designer’s schedule. Now someone who knows frontend (or an agent) can write it in HTML.

Three groups that should pay closest attention:

  • Content creators / influencers: Intros, title cards, data chart animations — write and edit your own, no dependency on designer schedules
  • Developers who regularly use AI agents: Install the skill on your agent, and the “describe video → get finished clip” workflow already works
  • People in product marketing / growth: Batch generate different versions of product intro videos for A/B testing, marginal cost approaches zero

People who don’t need to rush: teams with purely live-action needs, individual users with no frontend background who don’t use agents (the learning cost may be higher than just using CapCut templates).

One-sentence verdict: It’s not a “video tool” — it’s infrastructure that “merges video production capacity into the code production line”. For people already using AI to write code, this is free production capacity; for the content industry, this is another “skill translation”: web skills directly become video skills.

5. How to Get Started

Method 1: Install for your AI agent (recommended)

npx skills add heygen-com/hyperframes

After installation, just tell your agent:

Use /hyperframes to make a 10-second product intro: title fades in, background video, with light background music.

Note: For non-interactive/agent scenarios, use npx hyperframes skills update (to install the precise core set); the interactive picker with “select all” will install all 20 skills, which most people don’t need.

Method 2: Local CLI

npm install -g hyperframes

Requires Node.js ≥ 22. Write HTML (following its animation specifications), run the render command to get MP4.

Suggested learning path: First try the official Playground (hyperframes.dev) online to get a feel → then install the CLI locally to make your first clip → finally install the skill on your agent for batch production. The official docs have a Catalog (pre-built components like data charts) and Showcase (case library) — modifying existing examples is the fastest way to get started.

6. How to Build a Similar Solution Yourself

At the end of the day, hyperframes does three things: webpage rendering, frame-by-frame capture, and encoding into a finished video. Here’s the technical path if you want to build a working version yourself:

Layer 1: Frame-by-frame screenshots with a headless browser. Use Playwright or Puppeteer to open your HTML, adjust the time parameter at fixed intervals, and call page.screenshot() for each frame. The key is that your animations must be “time-parameter driven” — all state is determined by a single t variable. That’s the DIY version of “seekable”.

Layer 2: Frame sequence encoding. Use ffmpeg to combine the PNG sequence into MP4 (ffmpeg -framerate 30 -i frame_%04d.png -c:v libx264 out.mp4). This step uses mature tools, no gotchas here.

Layer 3 (optional): Wrap it in agent skills. Write the “video specifications” (dimensions, duration, animation conventions, no random numbers) into a SKILL.md file, so agents can write HTML according to the spec — this is where hyperframes’ real moat lies: it’s not that the code is hard, it’s that production specifications have been distilled into a process that agents can execute.

Where will a DIY version fall short compared to hyperframes? Determinism guarantees (they’ve done massive work to ensure the same frame always looks identical), media synchronization (audio track alignment), and rendering performance. So my advice is: use hyperframes first, and only consider building your own when you hit scenarios it can’t cover (like embedding into your own rendering pipeline) — unless your goal is to learn rendering fundamentals, in which case building one yourself is the best tuition.

Conclusion

The value of hyperframes isn’t in the trick of “turning HTML into video” — it’s that it picked the right level of abstraction: the essence of video is “interface state across a timeline”. Hand that essence to the most mature rendering engine (the browser) and the most widespread skill set (HTML/CSS), and all the remaining engineering problems become solvable.

For the average person, the most important takeaway from this piece is: content production capacity in the AI era is shifting from “knowing how to use a particular software” to “knowing how to describe what you want”. People who can write HTML get video production capacity for free; people who can’t, but use agents well, also get it for free — as long as you know this production line exists.

References