Unified multimodal models (UMMs) have opened up new possibilities for generating rich interleaved text-image content. However, they generally lack native support for coordinating the two modalities, and translating coarse user requests into coherent multimodal compositions also requires explicit planning. We propose UniWeaver, a structured, model-agnostic harness framework that integrates multimodal planning, generation, and visual feedback to produce coherent interleaved outputs. To train UMMs for this task, we construct UniWeaver-10K, a dataset of 10,000 tasks across 20 task families, paired with execution trajectories guided by UniWeaver. We also introduce UniWeaver-Bench, a task-oriented benchmark systematically assessing text coverage, writing quality, visual coverage, and image-text alignment. Experiments show that UniWeaver achieves state-of-the-art performance and improves interleaved text-image generation across multiple UMM backbones.
Problem setting and contributions
UMMs can both understand and generate, but they do not reliably coordinate a complete interleaved response. A coarse request still leaves the endpoint, number of units, and division between prose and visuals unspecified.
Six task families, one response each. Every unit pairs reader-facing prose with its own generated image, so a response can be read end to end.
Model-agnostic harness
Four planning stages turn a request into typed units, prompts, references and checks. Only an image that fails its check is regenerated, and no backbone parameter is updated.
UniWeaver-10K
10,000 goal-only tasks across 20 families, 500 per family, supervised at the plan level: no generated pixels and no explicit chain-of-thought as training targets.
UniWeaver-Bench
120 held-out tasks, 6 per family, scored on text coverage, writing quality, visual coverage and image-text alignment, with output validity entering each family score.
Sequential generation versus plan-first coordination
Sequential pipelines make each new unit depend on earlier outputs, even when the units are unrelated. UniWeaver fixes scope and modality roles first. Each image is generated from its own plan, and only failed units are retried.
In sequential generation each unit is conditioned on the previous output, so an early mistake carries forward; UniWeaver checks each generated image by default and regenerates only the ones that fail.
Four-stage planning with local visual repair
P1 sets scope, P2 separates prose from image prompts, P3 routes reusable references, and P4 defines observable checks. The model then renders each unit and repairs only images that fail a check.
Scope
Title, endpoint, and each unit's visual subject and content goal.
Text & Image
Reader-facing prose, image prompts, and one shared visual style.
References
An optional bank of at most two reusable references, routed only to the units that need them.
Checklist
One or two observable checks per image, phrased so a reader could confirm them by looking, for example whether the finished dish is plated.
Execution Generate each image → check it against its checklist → regenerate only failures, with at most two retries per image.
Planning, local repair, dataset construction, and output-level evaluation in one framework.
Goal-only supervision and output-level evaluation
Each training request gives only a goal. The model has to infer the unit count, order, and text-image split. The benchmark holds out 120 tasks from training and scores the completed response, not its internal plan.
Task distribution for both datasets; sample length and corpus vocabulary for UniWeaver-10K.
More than 30K goal-only candidates are screened for family consistency, naturalness, multimodal usefulness, format validity, and duplicate content. The final 10K set has 500 trajectories per family and a roughly 1:2:1 split across easy, medium, and hard.
Supervision covers planning outputs only. Generated pixels and explicit chain-of-thought are not training targets. Each trajectory follows the same four-stage schema used at inference, and later stages condition on earlier decisions.
The 120 held-out tasks cover four categories and six tasks per family. Each task supplies only a goal and hidden references. A response is valid only when it contains ordered text-image units, each pairing reader-facing text with an existing, non-reused image.
Each criterion is rated not met, partial, or met (0, 0.5, or 1). For every criterion, three judges each make two ratings at temperature 0, and the median is kept within each judge. A one-tier disagreement uses the lower tier. A disagreement spanning more than one tier scores zero. The family score is , where is quality on valid outputs and is the gate pass rate. The total is the mean of the 20 family scores.
Main results and ablations
The harness adds 31.2 points on BAGEL (27.7 → 58.9), 24.8 on OmniGen2 (27.9 → 52.7), and 21.3 on TokenFlow (39.2 → 60.5), with no parameter updates. Fine-tuning BAGEL on UniWeaver-10K lifts the same harness to 87.1, a further 28.2 points from adapting only the understanding decoder with LoRA while the image experts and vision encoder stay frozen.
Total score runs from 0 to 100. Baseline is a minimal step-wise harness. UniWeaver applies the harness without parameter updates. UniWeaver-SFT is the full harness on a BAGEL checkpoint fine-tuned on UniWeaver-10K.
BAGEL criterion scores use a 0 to 100 scale. Higher is better. SFT = UniWeaver-SFT.
Component ablations
Removing the task goal costs 5.88 points. Removing the shared visual style costs 10.68, and merging prose with image prompts costs 5.41. Reference choices contribute 0.41 to 1.21 points.
Qualitative comparison on three tasks
One row per system. Read across a row to compare coverage, repetition, and how each system renders text and images.
Evaluated tasks
BAGEL adds irrelevant accessories, OmniGen2 repeats checklist images, and Janus-Pro produces illegible in-image text. UniWeaver keeps clothing and equipment advice tied to a distinct image for each unit.
@article{chen2026uniweaver,
title = {UniWeaver: Harnessing Unified Multimodal
Models for Interleaved Text-Image Generation},
author = {Hongxu Chen and Zixuan Zhong and Donghao Zhou
and Haodong Li and Xiaomin Yu and Tianhao Li
and Shiyao Yu and Zhongwei Wan and Pheng-Ann Heng},
year = {2026},
note = {Project page}
}