Effect examples

1. Tutorial Introduction

EasyControl is a project designed to add efficient and flexible control to diffusion transformers. It was jointly developed in 2025 by Tiamat AI, ShanghaiTech University, National University of Singapore, and Liblib AI. Related research papers include... EasyControl: Adding Efficient and Flexible Control for Diffusion Transformer As diffusion model architectures shift from Unet-based structures to Diffusion Transformer (DiT), the existing DiT ecosystem lacks mature plugin support and faces issues such as efficiency bottlenecks, conflicts in multi-condition coordination, and insufficient model adaptability. To address these problems, EasyControl proposes an efficient and flexible unified conditional DiT framework. By introducing a lightweight conditional injection LoRA module, a location-aware training paradigm, and combining causal attention mechanisms and KV caching techniques, EasyControl significantly improves model compatibility, supporting plug-and-play functionality and lossless style control; it also enhances generation flexibility, supporting multiple resolutions, aspect ratios, and multi-condition combinations; and it optimizes inference efficiency, making the model more efficient at runtime.

This tutorial uses the stylized Img2Img control model, which is able to transform portraits into Hayao Miyazaki-style artworks. It was trained using only 100 real Asian faces and their corresponding Ghibli-style images generated by GPT-4o, while retaining facial features and applying the iconic anime aesthetic. In addition, EasyControl has collaborated with the CFG-Zero team to further improve image fidelity and controllability. The team also plans to continue to optimize model performance and explore more application scenarios. More pre-trained weights and training codes may be released in the future to support further research and development in the community.

This tutorial uses EasyControl as a demonstration, and the computing resource uses A6000.

2. Operation steps

1. After starting the container, click the API address to enter the Web interface

If "BadGateway" is displayed, it means the model is initializing. Since the model is large, please wait about 1-2 minutes and refresh the page.

2. Once you enter the website, you can start using

How to use

Demo

Exchange and discussion

🖌️ If you see a high-quality project, please leave a message in the background to recommend it! In addition, we have also established a tutorial exchange group. Welcome friends to scan the QR code and remark [SD Tutorial] to join the group to discuss various technical issues and share application effects↓