Abstract: In this paper we present the blue-c application programming interface, a software toolkit for media-rich, collaborative, immersive virtual reality applications. The blue-c API provides easy to use interfaces to all blue-c technology, including immersive projection, live 3D video acquisition and streaming, audio, tracking, and gesture recognition. The integration of multimedia data, including 2D video, 3D video, and animation, into the scene graph is presented. We emphasize on our performance-optimized 3D video handling and rendering pipeline, which is capable of rendering 3D video inlays consisting of up to 30,000 fragments updated at 10 Hz in real-time, enabling remote users to meet inside our virtual environment.