# How to Manage Robotics Training Data in 2026: Storage, Versioning, and Formats (LeRobot, HDF5, Zarr)

> Plan a robot-data pipeline across ROS 2, MCAP and LeRobot, with attention to timestamps, schemas, provenance and reproducible dataset conversion.

Source: https://hyperlake.cloud/blog/how-to-manage-robotics-training-data-in-2026-storage-versioning-and-formats-lerobot-hdf5-zarr
Published 2026-10-18 · by Hyperlake Team · Hyperlake

Managing robotics and physical AI training data requires a three-tier storage pipeline rather than a single file format: append-only **ROS 2 MCAP** (`rosbag2`) containers for zero-drop edge recording, a governed object-and-table lakehouse (**Apache Iceberg** and **Apache Parquet**) for episode curation and dataset versioning, and high-throughput training formats matched to your policy dataloader. In 2026, choose **Hugging Face **`**LeRobotDataset**`** v3.0** for standardized Vision-Language-Action (VLA) and imitation learning workflows with multi-episode Parquet and MP4 shards, **Zarr** for cloud-native chunked array streaming in Diffusion Policy and UMI pipelines, or **HDF5** (`h5py`) for low-latency random slicing on local NVMe storage.

*Disclosure: We build *[*Hyperlake*](https://hyperlake.cloud/)*, a sovereign AI operating platform. This reference guide reflects our engineering assessment and links directly to each open-source specification and vendor's official documentation.*

![Manage robotics training data: Record, Curate, Version, Reproduce.](https://hyperlake.cloud/blog/img/production/cc39712f4ab4f24754b1205cb8e1623c5c77c5c8-1600x960.png?w=1600&fit=max&auto=format)

*Manage robotics training data*

## Why Do Robotics Training Pipelines Need Multiple Data Formats?

Unlike text or static image datasets, physical AI trajectories combine heterogeneous sensor streams operating at different clock frequencies that must remain aligned within milliseconds [Trossen Robotics](https://www.trossenrobotics.com/post/robotics-dataset-formats-comparison). A typical bimanual manipulation or mobile robot logs:

- **High-frequency proprioception and control:** Joint positions, velocities, motor currents, and 6-axis force/torque wrench readings at 100 Hz to 500 Hz.
- **Multi-view visual streams:** Two to four RGB or RGB-D camera feeds operating at 30 to 90 FPS.
- **Spatial and navigation telemetry:** LiDAR point clouds, IMU packets, and odometry transforms (`/tf`).
- **Task and episode context:** Natural-language task instructions, embodiment calibration matrices, operator IDs, and success/failure labels [Encord](https://encord.com/blog/physical-ai-data-pipeline-collection-to-deployment/).

No single storage format optimizes for both ends of this lifecycle [Trossen Robotics](https://www.trossenrobotics.com/post/how-to-choose-robotic-data-pipeline-formats-libraries-physical-ai):

1. **Edge recording is write-bound and fault-sensitive:** Onboard robot compute must append asynchronous pub/sub messages to disk sequentially without dropping frames or corrupting the archive if a battery disconnects mid-episode.
1. **Dataset curation is query-bound:** Data engineers need columnar predicate pushdown to filter millions of trajectories by robot embodiment, lighting condition, task success, or collision recovery events without decoding terabytes of video.
1. **GPU training is random-read-bound:** Imitation learning and VLA dataloaders sample random $(t)$ indices across thousands of episodes and fetch temporal windows—such as observations at $[t - 0.2\text{s}, t - 0.1\text{s}, t]$ alongside an action horizon of $[t, t + 1, \dots, t + 16]$—requiring fast random seek and chunk decompression.

## Proposed Reference Architecture: From Robot Edge Recording to GPU Training

To prevent context loss across format handoffs and eliminate file-system bottlenecks, production robotics teams separate edge ingestion, lakehouse curation, and training consumption into distinct layers [Trossen Robotics](https://www.trossenrobotics.com/post/how-to-choose-robotic-data-pipeline-formats-libraries-physical-ai), [Encord](https://encord.com/blog/physical-ai-data-pipeline-collection-to-deployment/).

*Note: The diagram below illustrates a ****Proposed Reference Architecture**** combining open-source robotics formats with governed lakehouse and Kubernetes infrastructure.*

![Architecture connecting Tier 1: Robot Edge & Simulation Capture, Physical Robot Fleet (ROS 2 Nodes: 200Hz State + 60FPS RGB), rosbag2 Recorder (MCAP Storage Plugin + Zstd Chunks), Synthetic Generation (NVIDIA Isaac Sim / Isaac Lab), Trajectory Exporter, Tier 2: Governed Ingestion, Curation & Versioning, ("Object Storage (S3 / On-Prem MinIO)"), Alignment & Indexing Worker (Timestamp Sync & Schema Validation), Apache Iceberg Episode Catalog (Parquet Metadata + Trino SQL), Visual QA & Curation (Foxglove / Rerun), Dataset Versioning (Iceberg Snapshots / LakeFS / DVC), Tier 3: Sharded Training Stores & GPU Compute, LeRobotDataset v3.0 (Sharded Parquet + MP4 + meta/episodes), Zarr Store / Local HDF5 Cache (Chunked N-D Tensors), PyTorch DataLoader (delta_timestamps / Streaming), Distributed GPU Training (ACT / Diffusion Policy / VLA), Identity & Policy Control Plane, OAuth/OIDC Proxy + Scoped JWT + OPA Policy Enforcement.](https://hyperlake.cloud/blog/img/production/0a9aa1bb449499b607854ce471422af31a841114-1510x3600.png?w=1600&fit=max&auto=format)

*Proposed architecture; component selection depends on the workload.*

For a broader look at how these components deploy on Kubernetes, see our [Sovereign Physical AI Stack on Kubernetes: Reference Architecture](https://hyperlake.cloud/physical-ai/sovereign-physical-ai-stack-on-kubernetes) and the [Open-Source Physical AI Stack Map](https://hyperlake.cloud/physical-ai/open-source-stack-map).

## Robotics Data Formats Compared (2026 Consensus Matrix)

As of October 2026, five core formats and frameworks dominate physical AI data pipelines across open-source research and enterprise robotics deployments.

## How should formats be selected?

Use the format supported by the recording and training tools, and preserve conversion provenance. [LeRobot](https://huggingface.co/docs/lerobot/index) defines a training-oriented dataset structure; [MCAP](https://mcap.dev/spec) is a recording format. HDF5 and Zarr serve array-oriented workflows, while Parquet can hold tabular information. These roles can coexist in one pipeline.

Check the selected library versions, supported operations and recovery behaviour with representative data. No format choice removes the need to preserve timing, calibration and dataset lineage.

## How to Version and Curate Multi-Terabyte Robotics Datasets

Standard Git and Git-LFS break down when managing multi-terabyte robotics datasets containing thousands of MP4 shards, Zarr chunks, and raw MCAP logs. A production versioning architecture separates **binary blob immutability** from **catalog metadata snapshots**.

### 1. Immutable Object Storage + Open Table Snapshots

Write raw `.mcap` files and materialized training shards (`LeRobotDataset` v3.0 or `Zarr`) to object storage as **immutable, content-addressed or run-partitioned paths**. Rather than duplicating video shards when creating a new training split, maintain an **Apache Iceberg** episode catalog table (`robotics.episodes_catalog`) queried via Trino.

Each row in the Iceberg table tracks episode-level metadata and pointers:

- `episode_uuid`, `robot_id`, `embodiment_version`, `operator_id`
- `task_instruction`, `lighting_condition`, `outcome_status` (`SUCCESS`, `RECOVERY`, `COLLISION_ABORT`)
- `raw_mcap_uri` (`s3://robot-raw/2026-10/run_882.mcap`)
- `lerobot_shard_uri`, `shard_from_index`, `shard_to_index`

Whenever you curate a training dataset, query the Iceberg table at a specific **snapshot ID** (see [Apache Iceberg Snapshots Explained](https://hyperlake.cloud/blog/iceberg-snapshots-explained)). For teams that also want Git-like `branch`, `commit`, and `merge` semantics directly over raw S3 prefixes (such as branching a `LeRobotDataset` directory before running a data-cleaning job), tools like **LakeFS** or **DVC** provide zero-copy object pointer versioning, while **MLflow** or **Weights & Biases** binds the exact Iceberg snapshot ID or LakeFS commit hash to the trained policy checkpoint.

### 2. Preserve Failed and Recovery Trajectories

A common failure mode in imitation learning is deleting failed teleoperation runs during ingestion [Encord](https://encord.com/blog/physical-ai-data-pipeline-collection-to-deployment/). Policies trained purely on nominal expert demonstrations suffer from covariate shift the moment the robot drifts slightly off trajectory. Keep aborted, near-miss, and human-recovery trajectories in your object store, tag them in your Iceberg catalog (`outcome_status = 'RECOVERY'` or `'FAILURE'`), and filter them dynamically at dataset materialization time so they remain available for offline reinforcement learning, reward modeling, and safety evaluation [Encord](https://encord.com/blog/physical-ai-data-pipeline-collection-to-deployment/).

## How should storage and bandwidth be sized?

Measure representative recordings rather than applying a universal per-robot estimate. Camera settings, compression, sensor rates, episode length and retention all affect capacity.

Separate raw recordings, curated training data and temporary working files. Calculate growth and transfer demand for each, then include replication and recovery requirements. Validate the estimate against a real capture before buying capacity.

## Where Hyperlake Fits (and When to Use Standalone Open-Source Tools)

You do not always need an enterprise platform to manage robotics data:

- **When standalone open-source tools are the better choice:** If you operate a single academic lab arm or a small research team publishing public benchmarks, using the standalone `lerobot` Python library on local NVMe storage and pushing public datasets directly to the Hugging Face Hub is the fastest, lowest-friction path [LeRobot Docs](https://huggingface.co/docs/lerobot/en/index).
- **Where Hyperlake fits:** When physical AI teams transition from public demos to proprietary robot fleets, defense/industrial facilities, or customer sites, raw camera feeds and facility maps cannot be uploaded to public hubs. [Hyperlake](https://hyperlake.cloud/physical-ai) provides a sovereign AI operating platform running on Kubernetes in your own cloud (AWS, Azure, Google Cloud, OVH), private cloud, or on-premises environment with no compute markup.

Within a single governed foundation, Hyperlake lets teams assemble and operate:

- **Lakehouse Curation & Streaming:** Object storage paired with **Apache Iceberg** and **Trino** for SQL episode curation, **ClickHouse** for high-speed telemetry analytics, **Kafka** for streaming ingestion, and **Qdrant** or **Milvus** for embedding-based visual trajectory retrieval.
- **Identity-to-Data Governance:** People, training jobs, and agents access trajectory datasets through an OAuth/OIDC proxy, validated **JWTs** carrying scoped workload identity, and **OPA (Open Policy Agent)** policy checks logged at every access point (see [AI Data Governance: Lineage, Access, and Versioning](https://hyperlake.cloud/blog/ai-data-governance)).
- **Physical AI Simulation & Training Orchestration:** Reusable deployment patterns for simulation, training, data, and model serving (`KServe`), integrating tools such as **NVIDIA OSMO**, **Omniverse**, **Isaac Sim**, and **Isaac Lab** (subject to licensing and validated integration) alongside real-world robot feeds—as shown in our [live Physical AI Reliability & Safety Operations demonstration](https://live-demo.hyperlake.cloud/).

## Frequently Asked Questions

### How should a LeRobot dataset version be chosen?

Use the version supported by the selected training and recording tools. Check the [official LeRobot documentation](https://huggingface.co/docs/lerobot/index) and preserve the version with the dataset.

### Should I use HDF5, Zarr, or LeRobotDataset for robot imitation learning?

Use `**LeRobotDataset**`** v3.0** if you are training PyTorch VLA or imitation learning policies (such as ACT, SmolVLA, π₀, or GR00T) and want compact MP4 video sharding, built-in temporal windowing (`delta_timestamps`), and cloud streaming [LeRobot Docs](https://huggingface.co/docs/lerobot/en/index). Use **Zarr** if your pipeline (such as Diffusion Policy or UMI) streams pre-decoded multidimensional array chunks concurrently from S3 or GCS [Zarr Docs](https://zarr.dev/). Use **HDF5** (`h5py`) when your entire dataset fits on local NVMe storage and you want low-latency local array slicing without video codec decoding overhead [Trossen Robotics](https://www.trossenrobotics.com/post/robotics-dataset-formats-comparison).

### What should be recorded with a training dataset?

Record format and library versions, source recording identifiers, transformations, splits and relevant calibration. These details help another engineer reproduce the training input.

### Why should robots record in MCAP (`rosbag2`) instead of writing training formats directly?

Physical robots generate asynchronous sensor streams at different frequencies (for example, 200 Hz joint states and 30 FPS camera frames) and face sudden power disconnects. **MCAP** is an append-only container with indexed, chunk-level LZ4/Zstandard compression designed for structured recording [MCAP Docs](https://mcap.dev/). Writing training formats like HDF5 or multi-episode MP4/Parquet shards directly on the edge risks corrupting the entire file on power loss and requires real-time timestamp interpolation on constrained onboard compute [Trossen Robotics](https://www.trossenrobotics.com/post/how-to-choose-robotic-data-pipeline-formats-libraries-physical-ai).

### How do I version multi-terabyte robotics datasets without duplicating video files?

Store raw `.mcap` logs and sharded training files (`LeRobotDataset` v3.0 or `Zarr`) as immutable objects in S3-compatible storage, and manage dataset versions through an **Apache Iceberg** episode catalog table or an object-level versioning layer like **LakeFS** or **DVC**. When creating a new training split, you record a new Iceberg table snapshot or LakeFS commit pointing to the approved byte/frame ranges in existing shards rather than copying terabytes of MP4 or Zarr files.

*All trademarks belong to their respective owners. Notice an outdated format specification? *[*Contact the Hyperlake team*](https://hyperlake.cloud/contact)* and we will verify and update the documentation .*