tazmannner/Frieren-H3-LoRA

Frieren H3 LoRA (FRIEREN_H3_V1)

I do not own the rights, this guide is purely for educational and informative purposes and is in no way a means of substitution for the original work. All examples shown in this guide are generated using H3.

A rank-16 LoRA for MiniMax H3 that learned the look, cast and voices of Sousou no Frieren from 672 three-second dialogue clips and 721 stills, trained on images, video and audio together from step 0. This card is also the full build guide: how the data was cut, captioned, fixed, balanced, trained and tested.

Checkpoints

File	Step	Notes
FRIEREN_H3_V1-step00004200.safetensors	4200	Recommended. Most on-model faces and cleanest line work; lowest validation loss
FRIEREN_H3_V1-step00003900.safetensors	3900	The safe alternative; best fine-detail score, good Stark and Fern outfits
FRIEREN_H3_V1-step00003150.safetensors	3150	Softest palette; holds drawings on threes on still shots; follows lighting cues best

Put them in ComfyUI/models/loras/ and load them on the H3 model at strength 1.0. They were trained on the H3 FL2VA base and also work on the Ref2VA model with character sheets as picture references. Start every prompt with the trigger 2d anime style, Frieren style, (see Prompting the LoRA below).

Training data	672 clips (73 frames each, with audio) + 721 stills at 1920×1080
Validation	30 clips + 30 stills held out
Steps	4,200 (about 5.5 epochs), ~14 h on one RTX 5090
LoRA	rank 16, alpha 16, targets attention;mlp;audio

The super cut at step 4200 (the recommended checkpoint): two text-to-video scenes with no reference images, then same-prompt, same-seed wipes against base H3, voice A/Bs and the outro.

The pipeline

Every step from raw footage to a picked checkpoint, in the order it ran. Each step has its own section below.

1. Find shots: Scene-detect the footage on a small proxy and keep clean single shots.
2. Cut clips: Exact 73-frame windows at full resolution with audio, then verify each one.
3. Collect stills: The sharpest frame of selected clips, plus curated screenshots.
4. Caption: Gemini watches and listens to each clip and writes a three-line H3 caption.
5. Fix captions: Hand-labelled shot sizes, measured camera moves, face-checked character tags.
6. Balance and split: More Stark and other women, near-duplicates out, 30 + 30 held out.
7. Cache and train: One run, images + video + audio, save every 150 steps, validate every 300.
8. Test: Same prompt and seed on each checkpoint and on base H3, then the full Relay Hires finish.

Clips

Every clip is one shot, exactly 73 frames (3.04 s) at the native 23.976 fps, 1920×1080, with its audio. 73 sits on H3's 17k+5 frame grid, and about 3 s keeps training steps fast while still holding a full line of dialogue.

Six generated clips in the training format (3 s, one single shot each, muted here): Himmel, Frieren, Fern, Stark, Aura and Serie, rendered with the Ref2VA model, the LoRA and their character sheets.

How to cut them

1. Make a proxy. Downscale the footage to 270p with the same frame count, so detection runs fast and frame numbers still match full resolution.
2. Detect shots on the proxy and skip opening and ending credits, title cards and previews.
3. Pick a window. Drop 3 frames at each end of a shot (dissolve and bleed guard). If at least 73 frames are left, centre a 73-frame window in it; shorter shots are skipped.
4. Reject bad shots. A spike in frame-to-frame change inside the window (a flash, a missed cut, a dissolve) rejects the shot.
5. Spread the picks. Split the footage into time bins and take one eligible shot per bin, then review first / middle / last thumbnails on contact sheets.
6. Cut frame-exact. Seek a quarter frame early at full resolution and write exactly 73 frames: x264 CRF 14, 8-bit 4:2:0, audio as AAC 256 kbps. No fps conversion.
7. Verify every clip. Three PySceneDetect detectors must find no cut; frame count, duration and audio length must match; silent clips get a look.

The review thumbnails for one clip window (shown on a generated clip): first, middle and last frame. A cut or a dissolve shows up here as a different picture.

Stills

The 721 stills are full 1920×1080 PNG frames: 299 picked from clip frames and 422 hand-picked screenshots. Stills carry the line work and eye detail that 832×480 clips lose, so they train at 1376×768, a higher resolution than the clips.

Twelve generated stills in the training-still style, across the cast and scenery (LoRA renders, not dataset stills).

Picking a still from a clip (shown on a generated clip): score every frame in the middle half of the window by sharpness (Laplacian variance) and extract the sharpest one at full resolution as PNG.

Source	How it was picked	In the set
Frames from selected clips	The sharpest frame in the middle half of a clip's window	299 clip frames
Supporting-cast frames	Named side characters, including shots too short to be clips; about 30 per female character where the screen time allowed	(in the 299 above)
Curated screenshots	Hand-picked screenshot folders	422 screenshots
Environment screenshots	Unsorted scenery screenshots; any opening or ending credit frame removed by perceptual-hash match (distance ≤ 27)	(in the 422 above)

• A clip and its own still can both stay. One teaches motion and sound, the other a sharp single frame.
• Frieren was capped. 168 Frieren stills whose clip already stayed in training were moved out, so other women would not drift toward her face.
• Stills have no sound or motion. Their captions skip camera moves and set soundscape and music to N/A; the trainer never feeds stills to the audio loss.

Captions

Every item has a three-line caption in H3's own training format, written by Gemini 3.5 Flash Lite through OpenRouter at temperature 0.3. All 1,393 items plus the fix passes cost about $6.70.

1. integrated_multimodal_description: starts with [Shot 1] 2d anime style, Frieren style, and the shot size and angle, then one paragraph: who is there, their look, what they do, effects, background, light, the camera move and any dialogue.
2. overall_soundscape: 1–3 sentences of ambient and action sounds, no dialogue.
3. non_diegetic_music: 1–2 sentences on the score's instruments and tempo, no mood words.

Clip captions

Gemini gets each clip as a 640-px MP4 with its audio, so it hears and transcribes the Japanese dialogue. Recognised characters are written as a tag plus how they look in this clip; later mentions use the tag alone. Dialogue goes as char_X (S1) speaks … <d>[Japanese] …</d>, with plain punctuation only.

A generated example with sound: the same shot as this caption, rendered with the Ref2VA model and the LoRA, with Himmel's and Frieren's character sheets as picture references and a clean Himmel voice clip as the audio reference.

integrated_multimodal_description: [Shot 1] 2d anime style, Frieren style, close-up, over-the-shoulder, char_himmel, a young man with short light blue hair and blue eyes, wearing a white cape over blue and gold garments and carrying a heavy sword with a gold hilt on his shoulder, looks forward with a gentle smile. In the right foreground, the back of char_frieren, a petite elf girl with long silver hair in twin tails and a pointed ear adorned with a red drop earring, is visible out of focus. A stone wall and daylight illuminate the background. The camera slowly pulls out and slowly pans right. char_himmel (S1) speaks in a calm, reflective tone <d>[Japanese] 君のこの先の人生は、僕たちには想像もできない。</d>
overall_soundscape: Soft rustling fabric and a gentle outdoor breeze.
non_diegetic_music: A gentle, resonant string melody plays softly in the background.

Still captions

Stills use a separate prompt for one frozen moment: states instead of actions ("stands", "holds", never "turns" or "then"), no dialogue, no camera move. A word filter rejected any still caption containing speech or camera-motion words.

A generated example from this still caption (rendered as a short static clip, one frame shown).

integrated_multimodal_description: [Shot 1] 2d anime style, Frieren style, wide shot, low angle, a starry night sky filled with countless tiny white stars and specks against a deep indigo-blue background. Several bright comets streak diagonally across the frame, leaving long faint blue tails and glowing turquoise-blue heads with scattered white and pink debris trailing behind them. The lighting is dark and ethereal, illuminated by the soft glow of the celestial bodies.
overall_soundscape: N/A
non_diegetic_music: N/A

Fix passes

Gemini described scenes well but got shot sizes, camera moves and side-character names wrong. Each fix is a text-only Gemini edit followed by a word-change check that rejects any edit touching other words.

Shot size and angle: hand labels

Labelled by hand from contact sheets, then written in fixed wording.

Label	Written as
ECU · CU · MCU	extreme close-up · close-up · medium close-up
MS · MWS · WS · EWS	medium shot · medium wide shot · wide shot · extreme wide shot
INS	close-up insert shot
eye · low · high · overhead	eye-level · low angle · high angle · overhead top-down angle
profile · back · ots	profile view · view from behind · over-the-shoulder

Camera moves: measured

1. Track ORB features between frames at 480×270.
2. Fit a similarity transform with RANSAC, so moving characters fall out as outliers and the background drives the fit. A held drawing reads as no motion.
3. Sum the shift and scale over the clip and require a consistent direction across windows.
4. Write one sentence: The camera is a Static Shot., The camera slowly pans right.

Shake detection was tried and dropped as unreliable.

Character tags: face by face

Gemini tagged Heiter as Denken or Land, among others. The fix: detect faces with an anime face detector, crop each face that fills at least 6% of the frame height, and send the crops with a cast table filtered to that episode, so only characters who actually appear can be chosen. The caption is then edited to match, and a check rejects any tag outside the identified set. Recurring and supporting cast get tags; one-off characters keep a plain description.

Balance and validation

The training set is 672 clips + 721 stills. Frieren is in 41% of items and other women in 21%: plenty of Frieren, without every woman drifting toward her face.

Clips (purple) and stills (gold) per character tag. 79 clips and 150 stills have no tag (scenery, one-off characters); 628 of 672 clips have dialogue.

• More Stark and other women: face identification over unused shots added 72 clips and 71 stills.
• Supporting cast: 197 more stills of named side characters.
• Near-duplicates: only 15 true ones found and dropped. A clip and its own still do not count.
• Validation: 30 clips + 30 stills held out by near-duplicate group, so look-alikes went with them and the score measures unseen shots.

Training setup

The musubi-tuner fork's MiniMax H3 trainer, starting from the Golden Boy V3 image-and-video recipe with the audio settings from an earlier voice run.

Setting	Value	Why
Base model	H3 FL2VA, pruned, int8 ConvRot	The same base used in ComfyUI
Adapter	Ostris v3, training-only overlay	Replaces guidance distillation in training; not saved in the LoRA
LoRA	rank 16, alpha 16, attention;mlp;audio	audio adds the audio-only layers so voices can be learned
Optimizer	Automagic3, lr 1e-4, constant	The recipe from earlier runs
Clips	832×480, 73 frames, batch 1	About 11 s/step vs 13–14 at 1024×576; a higher-res run can start from these weights
Stills	1376×768, batch 8	Sharp detail; batching makes them cheap
Loss weights	video 1.0 · audio 0.2	Audio mattered less and must not overcook the run
Memory	block swap 20 (10 after resume), grad checkpointing, no compile	Mixed image and video shapes break compile; swap count did not change speed
Saves	every 150 steps; validate every 300, seed 42, 4 timestep bins	A fixed seed keeps scores comparable across resumes

What audio 0.2 really does with Automagic3

Automagic3 scales each layer's step by its own gradient history, so gradient size mostly cancels. The audio-only layers see only audio loss and still learn at full speed at 0.2. In the shared attention and MLP layers, 0.2 means each update is about 83% video and 17% audio, which keeps the visuals in charge. Dropping ;audio would have been the safest option; it was kept so the LoRA could learn the cast's voices, and audio never overcooked.

Dataset config (TOML)

[general]
enable_bucket = true
bucket_no_upscale = true

# Stills: images only, never audio
[[datasets]]
target_image_directory = "…/datasets/frieren_h3/images"
target_modalities = ["image"]
resolution = [1376, 768]
batch_size = 8
caption_extension = ".txt"

# Clips: video + audio, all 73 frames, 23.976 fps kept
[[datasets]]
target_video_directory = "…/datasets/frieren_h3/videos"
target_modalities = ["video", "audio"]
resolution = [832, 480]
batch_size = 1
caption_extension = ".txt"
frame_extraction = "full"
max_frames = 73
# source_fps unset so 23.976 clips load without resampling
# Validation config: same, frieren_h3_val folders, batch_size = 1 for both

Caching and train command

# Cache (both sets): video + audio latents, then text encoder outputs
python minimax_h3_cache_latents.py --dataset_config dataset_config.toml \
  --vae minimax_h3_video_vae_fp16.safetensors --audio_vae minimax_h3_audio_vae_fp32.safetensors
python minimax_h3_cache_text_encoder_outputs.py --dataset_config dataset_config.toml \
  --text_encoder qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors --text_encoder_quantization nvfp4_awq --task t2va

# Train
python -m accelerate.commands.launch --num_processes 1 --mixed_precision bf16 minimax_h3_train_network.py \
  --dit minimax_h3_fl2va_pruned_int8_convrot.safetensors \
  --dataset_config projects/frieren_h3/dataset_config.toml \
  --network_module networks.lora_minimax_h3 --network_dim 16 --network_alpha 16.0 \
  --h3_lora_targets "attention;mlp;audio" \
  --sdpa --int8_convrot_base --h3_convrot_int8_fwd bf16 --h3_fused_qk_norm_rope --h3_attn_auto_dispatch \
  --h3_overlay_weights minimax_h3_training_adapter_v3.safetensors --h3_overlay_training_only \
  --h3_batched_microbatch --h3_spatial_density_jitter 0.2 --num_timestep_buckets 8 \
  --h3_video_loss_weight 1.0 --h3_audio_loss_weight 0.2 \
  --optimizer_type Automagic3 --learning_rate 1e-4 --lr_scheduler constant --lr_warmup_steps 0 \
  --max_grad_norm 0 --max_train_steps 8000 \
  --blocks_to_swap 20 --use_pinned_memory_for_block_swap --block_swap_h2d_only --block_swap_ring_size 2 \
  --gradient_checkpointing --output_dir output/frieren_h3 --output_name FRIEREN_H3_V1 \
  --save_every_n_steps 150 --save_state --save_state_on_train_end --autoresume \
  --validation_dataset_config projects/frieren_h3/dataset_config_val.toml \
  --validate_at_start --validate_every_n_steps 300 --validation_seed 42 \
  --validation_timestep_bins 4 --validation_min_timestep 100 --validation_max_timestep 900 \
  --log_with tensorboard --logging_dir logs/frieren_h3

The training run

The run converged early: 84% of the total validation gain came by step 1500, and from step 3000 the scores moved by less than run-to-run noise. Nothing ever turned upward, so no overfitting and no overcooked audio.

Step	Overall	Video	Audio	Fine detail · bin_00	Composition · bin_03
0 (base)	0.2052	0.1838	0.3121	0.1441	0.3062
300	0.1938	0.1756	0.2845	0.1410	0.2848
1500	0.1908	0.1726	0.2822	0.1404	0.2782
3000	0.1899	0.1715	0.2818	0.1402	0.2757
3900	0.1897	0.1714	0.2815	0.1400	0.2756
4200	0.1896	0.1712	0.2815	0.1401	0.2755

• Caching took about 1.5 hours with ComfyUI stopped: stills in about 2 minutes, clips at 5.2 s each, captions through the 32B encoder at about 1.3 per second.
• Speed held at about 10.9 s/step. Swap 20 used 20–26 GB of VRAM; swap 10 used about 29 GB at the same speed. Each validation pass over the 60 held-out items took about 7.5 minutes.
• The bins split the score by noise level: bin_00 tracks fine detail, bin_03 composition and look. Composition moved first and most.
• Audio fell from 0.312 to 0.285 in 300 steps, then sat in a 0.2814–0.2820 band from step 2100: learned fast, then stopped.
• Validation could not rank 3300–4200. Each column's best landed on a different step, so the test renders decided.

Testing checkpoints

Against base H3, every checkpoint wins clearly on character fidelity and the show's look. Between 3150, 3900 and 4200 the gaps are small, and 4200 was the most consistent on faces. Every comparison uses the same prompt and seed, first pass only (euler, beta57, 20 steps at 960×544), and plays once per version with that version's audio.

Base H3 (left) vs step 4200 (right), same prompt and seed. Base follows the same shot plan, but Fern comes out off-model (brown eyes, a different outfit) and the demon grins like a horror villain.

Steps 3150 / 3900 / 4200 on the dungeon prompt. All three are clean; 3900 and 4200 get Stark's outfit and Fern's collar closer, and 4200 turns Frieren's eyes bluer.

Steps 3150 / 3900 / 4200 on the 15-second field opener. All lines come out right; 3900 and 4200 frame the point-of-view shot better.

Eyes at 3150 / 3900 / 4200. 3150 is the only one that casts the hand shadow across Frieren's face; 4200 has the calmest, most on-model eyes.

A character the LoRA never saw: base H3, 3150, 3900, 4200 (columns). The style carries over: thinner lines, longer face, calmer eyes, paler painted backgrounds.

The train platform with sound: base H3 and 3150 on top, 3900 and 4200 below.

Checkpoint	Val loss	What the renders showed
No LoRA	0.2052	Same shot plan, generic moe faces, thick lines, warm saturated colour; Fern off-model
1500	0.1908	Style and outfits far better than base; drawing holds no different from base yet (20–30% of moving frames vs ~70% in training clips)
3150	0.1900	Used for the first finals; holds drawings on threes when the camera is still (~92% on a sit-down shot); softest palette; follows lighting cues best
3900	0.1897	Better Stark outfit and Fern collar; best fine-detail score; the safe pick
4200	0.1896	Most on-model faces and cleanest line work; Frieren's eyes bluer and staff design varies in the dungeon test

Pipeline problems found and fixed

The full finish runs de-rope, FaceDetail, Anime Cadence (camera on follow) and the 1920×1088 relay. Testing the LoRA through it turned up four problems.

De-rope redrew Fern in her only close-up: first pass (left), de-rope final with red-brown eyes (middle), no-de-rope final keeping her purple eyes (right). Fix: render without de-rope and splice that shot in at its hard cuts.

But de-rope keeps action crisp: with it (left), the duck and the punch stay sharp; without it (right), Frieren smears. So it stays on for action and comes off for faces.

FaceDetail tore Frieren's bangs during a tilt-up (top, default 0.7 / 0.8 / 0.95). Lighter sigmas, 0.55 / 0.65 / 0.85, reduce it (bottom) without softening other faces.

Problem	Cause	Fix
White flashes in 1920 finals	taeh3 VAE on the relay step	Sol-H3 VAE for the relay encode and decode
A close-up face redrawn	de-rope on a face seen in one shot	Splice in the no-de-rope shot
Torn bangs during a tilt	FaceDetail redrawing a moving face too hard	Sigmas 0.55 / 0.65 / 0.85
A faint whisper mouth erased	relay strength 0.6	Relay sigma 0.45 for that clip (rare)
Choppy camera pans	Cadence holding drawings during moves	Camera mode follow, moves stay on ones

Finished scenes at step 4200

Text to video, no reference images, finished through the full pipeline at 1920×1088 (shown here at 720p).

Frieren vs a demon outside a pub: Stark and Fern in the foreground, a split line across a cut, a backward slide shot from behind, an over-the-shoulder ending. De-rope on, Fern's shot spliced from the no-de-rope render.

Frieren dunks on Stark, drops from the rim and boasts. De-rope on.

Fern and Frieren in a school hallway, with a low dutch angle, a rush past and a look back. No de-rope.

LoRA vs character sheets and voice references

The same cafe scene and seed four ways, first pass only: base H3 and the LoRA from text alone (FL2VA model), then base H3 and the LoRA on the Ref2VA model with Aura's and Frieren's character sheets as picture references and a clean voice clip for each as audio references. The video plays four times, once with each version's audio (the yellow AUDIO label marks whose sound is playing).

Aura. From text alone, with or without the LoRA, she comes out as a generic pink-haired demon in a strapless dress: the LoRA did not learn her well enough for likeness. The sheet fixes her hair colour, braids, horns, gold collar piece, gloves and wine-red skirt in both reference versions.

Frieren. Base H3 already knows her roughly but adds hair ribbons, misses the striped undershirt and draws thick lines. The LoRA alone gets the outfit and the thinner line work but turns her eyes blue. The sheet alone gets the outfit and eyes right. The LoRA plus the sheet gives the calmest, most on-model face with the show's line work and lighting.

Prompting the LoRA

Prompts work best when they read like the training captions.

1. Start with the trigger: 2d anime style, Frieren style, at the top of the description.
2. Tag, then describe, with nothing between. char_frieren, a petite elf girl with long white hair tied in twin tails, green eyes, pointed ears and red drop earrings, wearing …. A stray clause or a doubled comma between tag and description weakens the character, voice included.
3. Never put "elf" near Fern, not even "not an elf". Use char_fern, an ordinary human young woman with long straight dark purple hair that falls over the sides of her head and covers her ears completely, blunt bangs and purple eyes. The same wording keeps Flamme human.
4. Voice tags match the captions: char_X (S1) speaks with/in a … tone <d>[Japanese] …</d>. "Speaks", not "says". Borrow a character's common wording: Fern speaks softly, Frieren speaks in a flat, monotone voice, Flamme speaks in a warm, knowing voice.
5. Never repeat punctuation anywhere: no ……, ..., !! or ,,. One 。？！ per sentence in a line.
6. About 2–2.5 s of speech per shot. To carry a line across a cut, end the first half with 、 and put the rest in the next shot.
7. One shot per line: [Shot N] At 00:0X.X00, the shot cuts to ….
8. Pin who speaks. In a wide shot the voice can land on the wrong character. Tie the line to a face ("her lips move as she speaks", the other's "mouth closed") or cut to a close-up.
9. New actions change the staging. To change only the delivery, change only the voice wording.
10. Sell speed visually. Describe rushes, whips and gusts in the action; never write "slow motion".

Character sheets

Prompt guide: CHARACTERS.md lists every character's tag, prompt description, Ref2VA subject definition, voice wording, sheet and voice file, and how well the LoRA draws them from text alone (characters.json has the same data for scripts).

The picture references used above, one sheet per character: front, side and back views, a face and eye close-up, and the important accessories and main weapon shown separately (not held). They were generated with an image model from reference screenshots. Load a sheet in a LoadImage node on the Ref2VA reference slot and call it <Picture N> in the prompt.

Main cast, season 1 and 2 looks, regular and winter: Fern S1, Fern S1 winter, Fern S2, Fern S2 winter, Frieren S1, Frieren S1 winter, Frieren S2, Frieren S2 winter, Stark S1, Stark S1 winter, Stark S2, Stark S2 winter

Supporting cast: Aura, Denken, Edel, Ehre, Eisen, Flamme, Genau, Heiter, Hemmung, Himmel, Kanne, Kraft, Land, Lawine, Linie, Lügner, Methode, Methode (Alt), Richter, Scharf, Sein, Sense, Serie, Übel, Wirbel

Voice references: no audio from the show is shared. Instead, each character has a generated voice clip: about 7 s of new lines, rendered by H3 with the LoRA, the character's sheet and a voice reference, then cleaned of music with Mel-Band RoFormer. Use one as <Audio N> on the Ref2VA model. They are in the voices/ folder: Aura, Denken, Edel, Ehre, Eisen, Fern, Flamme, Frieren, Genau, Heiter, Hemmung, Himmel, Kanne, Kraft, Land, Lawine, Linie, Lügner, Methode, Richter, Scharf, Sein, Sense, Serie, Stark, Übel, Wirbel.

Conclusion

• H3 does not strictly need a character LoRA. A good character sheet and a clean voice reference already get the general look and voice of a character, even on the base model.
• A style LoRA is what makes it look like the show. The colours, line work, motion and animation timing are greatly improved by the LoRA, and the references cannot supply those.
• Characters mostly came through only where the base model already knew them. Frieren, Fern and Stark are solid; a less-seen character like Aura was not trained well enough for likeness, so picture references and voice references are needed for most of the cast.
• The show's music and sound were trained too. The LoRA produces lovely Frieren-style background music and ambience, which is worth having on its own.

So the LoRA brings the colours, animation and sound of Frieren, and the references fill in the character details.