Data collection is only half the battle. Our dedicated team of 50+ data engineers and QA specialists inspects, time-synchronizes, transcribes, annotates, and formats every single episode for direct ingestion into your model training loop.
How our data team turns multi-camera feeds and human demonstrations into production-grade training tokens:
Raw 4K video feeds, EgoCAM streams, audio recordings, depth frames, and robot joint states are ingested over 100GbE optical fabric. Every frame is aligned to the IEEE 1588v2 hardware PTP clock with <0.08ms measured shutter jitter.
Spoken step-by-step human narrations are transcribed with high-accuracy speech models and audited by human verifiers. Each sentence, phrase, and sub-goal is given millisecond-accurate start/stop timestamps mapped to the exact action frame.
Episodes are segmented into discrete semantic primitives (e.g. reach_carton, grasp_tape, fold_flap). Optional 21-keypoint 3D MANO hand tracking, 3D bounding boxes, and object masks are synthesized.
Every recorded episode passes three independent gates before approval: 1. Automated kinematic limit & frame-continuity checks. 2. Senior data annotator visual audit for occlusion and task fidelity. 3. Engineering Director random sampling sign-off.
Datasets are compiled into target schemas: Open X-Embodiment RLDS (TFDS format), standardized HDF5 with LZ4 compression, Hugging Face lerobot format, or custom client WebDataset tarballs with PyTorch/JAX DataLoaders.
Client data is isolated on private VLANs with physical encryption at rest (AES-256). Datasets are dispatched via private AWS S3 buckets, Snowflake share, or physical encrypted enterprise SSD drop-ships under strict MNDA.
Plug our datasets directly into OpenVLA, π0, Octo, ACT, and Diffusion Policy pipelines:
import torch
from torch.utils.data import Dataset, DataLoader
import h5py
class SkyAiMultimodalDataset(Dataset):
"""
Sky AI LAB — Multimodal Human Demonstration & Narration Loader
Time-synchronized 4K EgoCam + Audio Tokens + Bimanual Joint Trajectory
"""
def __init__(self, h5_path: str, chunk_size: int = 16):
self.f = h5py.File(h5_path, 'r')
self.chunk_size = chunk_size
self.images = self.f['/observations/images/ego_head'] # (N, 3, 224, 224)
self.narration = self.f['/language_instruction'][:] # Text instruction tokens
self.actions = self.f['/action'][:] # (N, 14) Bimanual actions
def __len__(self):
return len(self.actions) - self.chunk_size
def __getitem__(self, i):
return {
'image': torch.tensor(self.images[i] / 255.0, dtype=torch.float32),
'action_chunk': torch.tensor(self.actions[i:i+self.chunk_size], dtype=torch.float32),
'instruction': str(self.narration[i])
}
# Batch loading ready for OpenVLA / ACT imitation policy fine-tuning
loader = DataLoader(SkyAiMultimodalDataset('skyai_packaging_task_088.h5'), batch_size=32, shuffle=True)