FlowAct-R2: Beyond Talking Avatar via Streaming Multimodal References and Proactive Agent Planning
What it is
FlowAct-R2 is a framework for interactive humanoid video generation that integrates a Streaming Multimodal Reference Diffusion Transformer with a Proactive Interaction Agent. It enables real-time, hour-scale streaming video by combining continuous multimodal control (action prompts, audio, and dynamic references) with autonomous planning for persona management and audience interaction.
The problem it solves
Existing interactive video generation methods struggle with accumulated drift over long durations and lack the proactive planning capabilities needed to handle complex, real-time audience interactions and interruptions.
What you could use it for
- Entertainment streaming with interactive avatars
- Live shopping with responsive virtual hosts
- Real-time video chatting with proactive agents
- Live vlogging with autonomous behavior scheduling
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.