How AI Lip Sync Makes a Cartoon MC Rap Your Words

On DissKO, you type one sentence and after a short wait a cartoon MC raps eight bars about it, on beat, with a mouth that moves along with every word. It can look like one piece of magic. It's actually three separate AI systems handing work to each other, each doing one job.

This post explains that pipeline in plain terms: what each stage does, where the waiting time comes from, why every verse is 15 seconds, and why the performers are original cartoon characters instead of real faces. We won't name vendors or claim specific benchmarks. The goal is to explain how the process works, not to sell a particular model.

The short version

Every verse goes through three stages:

1. Lyrics. A large language model turns your opinion into an 8-bar rap verse.
2. Audio. A music generation model turns those lyrics into vocals sung over a beat.
3. Video. A lip-sync model takes a still reference image of the MC and animates it so the mouth and face match the audio.

Each stage depends on the one before it. You can't make the audio without lyrics, and you can't animate a mouth without audio to follow.

Stage 1: From opinion to lyrics

The first stage is text in, text out. You type something like "Cats are low-maintenance roommates who never ask you for a walk in the rain." The language model's job is to turn that into eight bars that:

• Keep your argument intact
• Follow a rhyme scheme
• Fit a rhythm that can be rapped in about 15 seconds
• End on, or build to, a punchline
• Stay in a PG-13 tone

Context makes the verse better

In a battle, the model doesn't only see your latest opinion. It also has the topic, your side, and what your opponent said in their last verse. That context is how your verse ends up answering the other player instead of floating on its own. It's also why relevance is one of the four things the AI judge scores, alongside rhyme, flow, and punchline.

Why the lyrics are constrained

Eight bars is a fixed target. A free-form model might write sixteen lines, or a paragraph, or a verse with syllable counts all over the place. Those would be hard to sing in the time available. So the lyrics stage works inside tight limits on length and structure, which makes the next stage much more reliable.

This stage is also where safety rules apply first. Disses target the opponent's side and opinion, not a real person's identity or appearance. The lyrics are written to those rules before anything becomes audio.

Stage 2: From lyrics to vocals and a beat

The second stage takes the finished lyrics and produces audio: a voice rapping the words over an instrumental. Modern music models can generate vocals and backing music together from text input plus a description of the style.

What the music model has to get right

Rap is demanding for generated audio. The model has to:

• Pronounce every word clearly, even at speed.
• Place syllables on the beat, so the flow feels intentional.
• Keep a consistent voice across the whole verse.
• Fit the time window, so the verse doesn't get cut off or trail into silence.

When the lyrics from Stage 1 are already shaped for a 15-second delivery, the music model has a much easier job. Good input structure is the main reason flow comes out clean.

No voice cloning

The voices on DissKO are generated, not copied. There's no upload of your own voice and no imitation of real rappers. This matters both for safety and for keeping the product clearly in the "original character" lane.

Stage 3: From audio to a lip-synced video

The third stage is the one people notice. A lip-sync model takes two inputs:

1. A reference image of the MC, a single still picture of an original cartoon character.
2. The audio from Stage 2.

From those, it generates a sequence of video frames where the character's mouth shapes match the sounds in the audio. Better models also move the head, eyes, and expression in time with the rhythm, which makes the character look like it's performing rather than just talking.

How audio-driven animation works, broadly

The general idea behind audio-driven lip sync is that sounds map to mouth shapes. An "oo" sound means rounded lips. An "m" or "b" means closed lips. An "ah" means an open jaw. The model learns those relationships from large amounts of video with matching audio, then applies them to a new face frame by frame.

For a still image, the model also has to invent motion that wasn't there: the head nods with the beat, the shoulders shift, the eyes blink. That's what turns a static drawing into a performer.

Why rap is harder than speech

Talking-head lip sync for normal speech is a well-studied problem. Rap adds difficulty:

• Speed. More syllables per second means more mouth shapes to hit precisely.
• Rhythm. Delivery stretches and compresses words to fit the beat, which changes how mouth shapes are timed.
• Music in the mix. The model has to follow the vocal, not the drums or the bass line.

These are the reasons cartoon MCs can still show the occasional slightly off frame. The overall effect holds up best when the verse is short and the vocal is clear, which loops back to the design choices in the first two stages.

Where the waiting time comes from

Generation isn't instant, and it helps to know why.

Lyrics are fast

Text generation for eight bars is the quickest stage. It usually finishes in seconds.

Audio takes longer

Generating singing over a beat is heavier work than writing text. It typically takes noticeably longer than the lyrics.

Video takes the longest

Lip-sync video is the slowest stage by far. The model generates many frames per second of video, and every frame has to stay consistent with the reference image and with its neighbors. That's a lot of computation per verse.

Resolution affects speed

DissKO offers 480p verses for 1 credit and 768p verses for 2 credits. Higher resolution means more pixels per frame to generate, which takes more computing time. That's why it costs more. The lyrics and audio are identical at both resolutions.

When something fails

Any of the three stages can occasionally fail, for example if a model times out. When that happens, DissKO refunds the credit automatically. You don't need to contact anyone or file a request.

Why every verse is 15 seconds

Fifteen seconds isn't arbitrary. It's a balance between several constraints.

It fits eight bars

At a typical rap tempo, eight bars of lyrics fit naturally in roughly 15 seconds. That's long enough for a setup, a few supporting lines, and a punchline.

It keeps the battle moving

DissKO is a turn-based game. While one player's verse generates, the other player is waiting. A shorter verse keeps that wait reasonable, so the back-and-forth still feels like a conversation instead of a queue.

It keeps quality consistent

Longer generated audio and video give the models more room to drift: a voice that changes slightly halfway through, or a face that slowly loses its shape. Short clips stay closer to the reference throughout.

It's easy to watch and share

A full battle is six verses, about a minute and a half of rap. That's short enough that spectators in the arena watch the whole thing, and short enough for a 9:16 vertical download to work on short-form video apps.

Why DissKO uses original characters

The MCs on DissKO are original fictional cartoon characters. You can't upload a selfie, you can't clone a voice, and none of the characters are designed to look or sound like a real rapper. This is a product decision, not a technical limitation.

Avoiding real faces avoids real harm

Lip-sync models can, in principle, animate any face. Pointed at a real person, that becomes a tool for putting words in someone's mouth. Limiting the reference images to original characters removes that risk entirely.

Cartoons handle stylization well

A cartoon face has fewer fine details than a photo of a real person. Small imperfections in mouth shapes are less jarring on a stylized character, which makes the overall performance read better.

Characters give battles personality

Each MC has a distinct look, which gives each player a visual identity in the battle. Spectators know which verse belongs to which side at a glance.

Every video is labeled

Every DissKO video carries an "AI generated" badge. Even with original characters, it should always be clear to viewers that what they're watching was made by AI.

Putting it together in a battle

Here's the whole flow for one turn:

1. You type an opinion that answers your opponent.
2. The language model writes eight bars, using the topic, your side, and the last verse as context.
3. The music model sings it over a beat in about 15 seconds of audio.
4. The lip-sync model animates your MC from that audio.
5. The finished video plays for both players and any spectators.
6. The AI judge scores the verse on rhyme, flow, relevance, and punchline.

Three rounds of that, alternating, and you have a battle. Spectators who sign in with Google vote once per account, and the vote is shown next to the AI score without being merged into it.

After the battle, the replay page shows every verse with lyrics as text, scores, and options to share, embed, or download a 9:16 video with a dissko.fun watermark.

Try it yourself

The fastest way to see the pipeline is to run it. Drop your take to get first in line. New users get 1 free credit on Google sign-in for a vs-AI battle. For more on the video side, see AI rap lip sync, or read how to make an AI rap battle video for a step-by-step walkthrough.