Physical AI needs a new datastack - how we built it скачать в хорошем качестве

Physical AI needs a new datastack - how we built it

The unique value of Rerun is rooted in its data representation and data format. The core components discussed are: 00:00 Block-sparse Columnar Storage 00:30 Powering flexible and fast queriying/ingestion   * *File Format* Rerun uses a new, column-based file format that is designed for physical data and optimized for flexible, high-performance processing.       * This format is built for multi-rate data, which necessitates it being sparse.       * The core is block-sparse columnar storage.       * This design powers the high performance, flexibility, and fast querying/ingestion in the visualizer, especially for physical data.       * It is used in a data lakehouse architecture for the cloud data platform to work with large sets of robotics data.       * In the open-source version, it helps query, analyze, and visualize smaller data sets flexibly. 01:00 Entity-Component System Data Model 01:30 Composable data for evolution over time   * *Data Model* Built on top of the core format is a data model that describes the data.       * It draws inspiration from entity-component systems found in the game engine world.       * This results in a more composable data model.       * In robotics, data models typically stop at the message level. For example, a robotics message might be an image buffer with some metadata.       * If you wanted to add extra data to that message on the fly—such as a segmentation map, labels, or output from a neural network—you need a data model that allows for more fine-grained composability.       * This modular, composable data model allows queries and the visualizer to make sense of the data. 02:00 High-performance visualization   * *Visualizer* The visualizer is the last piece and is considered super important.       * Users require super high performance and flexible visualization.       * It is designed so that people can embed it in their own tools, use it in a quick, scrappy script within a notebook, or use it to build central visualization dashboards.