Rerun

Rerun is an open-source data layer designed specifically for the field of Physical AI and robotics. This tool functions as a complete ecosystem that allows users to log, transform, query, and visualize multimodal and variable-frequency data. Its primary goal is to facilitate the creation of efficient data loops—from the initial recording phase to large-scale model training—by providing a highly flexible and extensible open-source SDK.

Rerun's architecture stands out for its high performance and versatility, offering native APIs for Python, Rust, and C++. It utilizes a data-primitives-based approach that allows developers to integrate the tool into their existing workflows without needing to migrate to closed platforms. Thanks to its compatibility with WebAssembly (Wasm), the interactive visualizer can run both on a desktop and directly in the browser, providing quick access to visual representations of sensors and system states in real time.

Key functional capabilities include the ability to perform SQL and dataframe queries directly on robotics recordings. This allows for filtering and processing information by time ranges or specific values without performing heavy data copies. The storage system utilizes .rrd files, which organize information into columnar blocks, optimizing read speeds and conversion from other formats.

Its core features include:

• Multimodal Visualization: Review complex datasets, debug issues at a granular level, and create custom views for every development stage.
• Physical AI Debugging: Simplify the diagnosis of calibration errors, trajectories, and misalignments in 3D systems and computer vision.
• Direct Training: Stream data directly to the GPU, eliminating intermediate export steps and optimizing resource usage.
• Data Transformation: Add derived columns and evolve schemas without breaking history, maintaining the consistency of the original data.

This tool is used by engineering teams in cutting-edge projects, such as Hugging Face’s LeRobot or Meta’s Project Aria, for training inspection and egocentric data sequence visualization. For environments requiring scalability, the platform offers Rerun Hub, a managed backend that adds centralized catalog features, byte-range indexing, and S3-compatible cloud storage. By integrating Rerun, developers of artificial intelligence applied to the physical world achieve total failure traceability and a significant acceleration in the model development lifecycle.