Lingen Li Guangzhi Wang Zhaoyang Zhang Yaowei Li Xiaoyu Li Qi Dou Jinwei Gu Tianfan Xue Ying Shan

Abstract

Traditional cartoon and anime production involves keyframing, inbetweening,and colorization stages, which require intensive manual effort. Despite recentadvances in AI, existing methods often handle these stages separately, leadingto error accumulation and artifacts. For instance, inbetweening approachesstruggle with large motions, while colorization methods require dense per-framesketches. To address this, we introduce ToonComposer, a generative model thatunifies inbetweening and colorization into a single post-keyframing stage.ToonComposer employs a sparse sketch injection mechanism to provide precisecontrol using keyframe sketches. Additionally, it uses a cartoon adaptationmethod with the spatial low-rank adapter to tailor a modern video foundationmodel to the cartoon domain while keeping its temporal prior intact. Requiringas few as a single sketch and a colored reference frame, ToonComposer excelswith sparse inputs, while also supporting multiple sketches at any temporallocation for more precise motion control. This dual capability reduces manualworkload and improves flexibility, empowering artists in real-world scenarios.To evaluate our model, we further created PKBench, a benchmark featuringhuman-drawn sketches that simulate real-world use cases. Our evaluationdemonstrates that ToonComposer outperforms existing methods in visual quality,motion consistency, and production efficiency, offering a superior and moreflexible solution for AI-assisted cartoon production.