Project Overview

LiveCC was first released on April 25, 2025 by the Show Lab of the National University of Singapore and ByteDance. LiveCC is a video language model project focusing on large-scale streaming speech transcription. The project aims to train the first video language model with real-time commentary capabilities through an innovative video-automatic speech recognition (ASR) streaming method, achieving the current state-of-the-art (SOTA) level in both streaming and offline benchmarks. The related paper results are "LiveCC: Learning Video LLM with Streaming Speech Transcription at Scale", which has been included in CVPR 2025.

This tutorial uses a single RTX A6000 card as the resource.

Project Examples

Run steps

1. After starting the container, click the API address to enter the Web interface

2. Once you enter the web page, you can interact with the model

If "Bad Gateway" is displayed, it means the model is initializing. Since the model is large, please wait about 1-2 minutes and refresh the page.

This tutorial provides two module tests: Real-Time Commentary and Conversation modules.

Do not switch models frequently to avoid resource congestion.

The functions of each module are as follows:

Real-Time Commentary

Exchange and discussion

🖌️ If you see a high-quality project, please leave a message in the background to recommend it! In addition, we have also established a tutorial exchange group. Welcome friends to scan the QR code and remark [SD Tutorial] to join the group to discuss various technical issues and share application effects↓

Citation Information

The citation information for this project is as follows:

@inproceedings{livecc,
    author       = {Joya Chen and Ziyun Zeng and Yiqi Lin and Wei Li and Zejun Ma and Mike Zheng Shou},
    title        = {LiveCC: Learning Video LLM with Streaming Speech Transcription at Scale},
    booktitle    = {CVPR},
    year         = {2025},
}